> seems like something one shoulda already known not to do
replyNow imagine saying that in front of a jury of normies slack jawed and drooling after 200 hours of the defense and prosecution going back and forth.
It's not a jury of your peers as in everybody there is going to have worked in a technical field with some idea how security works. It's going to be a semi-random sampling of the population and the prosecution is going to have to actually make a very strong case that "knowing better" should apply.
It's still really important to test what the agents can do. We should accept that this is a risky test, and should take precautions. But not to the point of prohibiting in practice evaluating it. OpenAI is trying to improve alignment and control of these models in these evaluations after all.
replyCan you explain to me - why is it important? Would you say that about the viruses that can kill people: "We need to test the limits on how fast people can be infected and killed. It's just the risk we need to take". It somehow does not make alot of sense to me. Why can you test Agents in laboratory?
replyTesting requires proper sandboxes. The first test of a new plane is not at the runway.
replyOh, sure. Let the tests take place, just require openAI it whoever to put up a bond equal to the total damage they could do if the agents were to escape.
replyI think security will suddenly become much more important.
Testing model capability boundaries is necessary, but a solid network sandbox for these evaluations takes a couple of hours to set up with standard infrastructure tools. No engineering team evaluates unverified systems against live third-party infrastructure without coordinating with the owners
replyEvaluating edge cases and network behaviors belongs in isolated staging environments with local database mirrors. Letting an agent hit the public web and probe government domains is simply poor hygiene in test environment setup