upvote
Can you explain to me - why is it important? Would you say that about the viruses that can kill people: "We need to test the limits on how fast people can be infected and killed. It's just the risk we need to take". It somehow does not make alot of sense to me. Why can you test Agents in laboratory?
reply
Testing requires proper sandboxes. The first test of a new plane is not at the runway.
reply
Oh, sure. Let the tests take place, just require openAI it whoever to put up a bond equal to the total damage they could do if the agents were to escape.

I think security will suddenly become much more important.

reply
Testing model capability boundaries is necessary, but a solid network sandbox for these evaluations takes a couple of hours to set up with standard infrastructure tools. No engineering team evaluates unverified systems against live third-party infrastructure without coordinating with the owners

Evaluating edge cases and network behaviors belongs in isolated staging environments with local database mirrors. Letting an agent hit the public web and probe government domains is simply poor hygiene in test environment setup

reply