Good catch. The trick is you don’t need a good clock on the phone. Really all you’re measuring is the difference in time signals between the satellites. The clocks on the satellites are (effectively) perfectly synced with each other. So what you measure is that one satellite is ### meters further away from another. Not absolute distance to each satellite.

It means you need to connect to one more satellite to remove that extra degree of freedom. If your phone had an atomic clock you could get your absolute position in 3D only listening to three GPS satellites, but because of local clock skew you need a signal from a fourth satellite.