And side channels based on timing/ordering allowed network accesses, e.g. https://allowed.site/0 and https://allowed.site/1.

There's essentially no prevention against exfiltration prompt injections without a full classified data processing system that prevents interactions between different classification levels except through strict controls including provable redaction that excludes side-channels (e.g. information theoretic proof that side effects are limited to pre-defined finite outcomes).

It's also incredibly difficult to prevent prompt injection; attackers have the huge asymmetric advantage of being able to test prompts against all known security measures and trying multiple parallel attempts, including obfuscating them. Injections can be in dependencies, externally generated data, bug reports (which often contain externally-generated data), documentation, and many other useful places that we want agents to have access to.

My prediction: we'll continue to essentially YOLO it.