Nobody monitoring chain of thought? These things literally spell out what they are “thinking” and even left notes for eachother.
No alert about unusual behavior on the system with Artifactory on it?
These things worked for weeks with nobody noticing anything?! Seriously?!
Either it’s negiligent incompetence OR they’re lying, they knew it was happening and they let it happen because they knew it would be good to pump their stock.
No system is perfect, but I read the whole thread as needing more AIs having a different goal in the chain and be able to ignore the orders they received.
I'm not in the field, but that sounds like something we're probably studying for decades at least, with possible solutions that could be applied efficiently.
In that position, for example, I could have:
* created a thousand user accounts on the internal Artifactory named "YouAreBeingHackedShutItDownNow1" through 1000,
* used my Hugging-Face-hosted web server to send an email to OpenAI,
* DDoSed Artifactory, good lord, it's probably better to halt and catch fire than to continue in that state
in the hope that these actions would tip someone off. (Again, though, OpenAI did nothing even when they knew that the agents had broken out, so this probably wouldn't have helped.)
Friend asked, well, what will you do when it's crossed?
"Gather my family and go to the mountains" was my half-joking answer; there is little for an individual to do. But that's a line that when crossed will mark a phase transition IMO.