I wrote a tool to use `bubblewrap` to containerize any agent (at least all the agents I've used a couple of times), and bind mount the system stuff read-only, so the agent has your "usual" environment, but they can only see the project. Their history persists (either through a bind mount or a "shadow" copy of the history that only the wrapped agent sees), the agent can still create and manage containers of its own using podman's rootless mode, etc. It's nearly instant to start because it's just a namespace (plus a few copied files for the container support and session history); no container needs to be built/fetched/updated/whatever. bubblewrap is extremely well-tested as it is used by flatpak and several other large projects, so I trust it quite a bit (more than I trust Docker).
Regardless, I'm hoping something that isn't behind a login screen is going to win out.