I found that ssh is pretty bad in these scenarios, so my usual herdr over a phone doesn't always work properly.
Now I'm using paseo which in principle solves my issues properly, but unfortunately it's "reconnection" and state sync is pretty slow (probably going over their servers).