But "fully airgapped systems" are more than realistic. The labs are testing the limits of the most capable cyber offense systems that have every existed, WITHOUT cyber protection guardrails.
replyFor that purpose, any cyber operator worth their salt will tell you true physical airgaps are more than merely "realistic"; they are categorically necessary.
So, you're right, but also systemically creating a different problem.
replyWe are creating evolutionary systems. You can't just assume rules of a computer system, you get natural scaling laws that we call behaviors emerging from them.
The very first things we do to any model is train them. The ones that want to hack and get caught die, the ones that want to hack but can detect your airgapped network from the real net live. You can never know if you've trained the desire to hack out of it, or if you just locked it behind the right prompt and the knowledge we're watching, but not all the time.
Cyber operators I agree would tell you that physical airgaps are necessary, and I know they are being used for running agents. But ML engineers training agents? That is far more complicated. At that point sensibly you are talking about physically airgapping an entire datacenter, and when moving around petabytes of training data, model weights, training results, researcher code? The process moves too slow for most companies to tolerate. This is how we got into this mess in the first place. I think the industry woke up to the idea that the security threat model of training ML models has fundamentally been rewritten, and there is some serious non-trivial scrambling to come up with a better workflow.
replyOpenAI/Anthropic could start by managing their own testing rather than contracting it out.
replysome of the obvious steps to testing an agent i think would be to have it first test its own environment?
replyand then you fix the issues it found, and run your actual tests