Note that, once you're inside the horizon, you can't "turn around" and go back outside again. You're inside the hole for good.
And once you're inside the hole, yes, no matter which direction in space you move, you're moving "towards" the singularity. But a better way to look at it is that the singularity is a moment of time, not a place in space. You're moving "towards" the singularity in the same sense as you're moving "towards" next Tuesday. You can't stop moving towards next Tuesday by changing which direction in space you move. The same is true for the singularity once you're inside the hole's horizon.
Unless you went in butt-first, but the path of the photons would have changed and would now be going toward the black hole, and everything would look probably all smushed together.
You can avoid a coordinate, for example by choosing not to go there, or revisit another one repeatedly.
A black hole on the other hand doesn't have that: you cannot revisit old locations - attempting to do so moves you closer to the singularity.