Why not use vscode remote via ssh port forwarding? I just use it via my browser.

You can run it directly from a web browser too.

There is ssh extension and remote containers extension, they work together.