>It's also hardly outrageous that they ran training and/or benchmarks with only network-isolated VMs with access to a package repository.
At least in the black hat talk they gave, they said that they noticed that the agents had compromised the package repo, and they remediated it, patched the vulnerability the agents had used, then continued. But given that it was compromised again and they didn't detect it the second time, clearly they did not add monitoring or hardening or comprehensively audit it for further vulnerabilities. My point is that even if there had not been a widely known incident like this yet, that only could excuse the first artifactory compromise, not the second. They had notice because it had already happened to them and they detected it.
As for breaking out of the VM, my point is that the agent should have had to. The comment
>they were running in network-isolated VMs with no access to the internet.
Was not accurate, the network isolation was not done at the VM level.
Not sure what you mean with the network isolation not being at the VM level. I'm getting a bit lost here in the details of where what network isolation was applied, and the structure of the container runtime and what the apparently compromised parent VM refers to.
In any case, I understand that after the HF incident was found and they realized the danger, they removed Artifactory and switched to two separate layers of network isolation.
That said, I believe it was compromised again last week:
> OpenAI describes an internal research model that, during RL training on September 20, exploited insufficient DNS filtering in its sandbox to contact a public chatbot service. OpenAI’s monitoring flagged the behavior within 15 minutes, a human reviewer acknowledged it three minutes later, and the run was killed about 2.5 hours after the external contact
Apparently they have now paused training and inference for their most capable model because of this.
>OpenAI also required all Research CaaS workloads to use a micro VM sandbox
So yeah, they were not using it before and they should have been, and they're doing it they way they should have, now.
>exploited insufficient DNS filtering in its sandbox
If your goal is to use dns filtering to prevent your models from reaching the open Internet then you are not really serious about security. People have been tunnelling DNS to get internet access on captive portal wifi since 1998. There's not a lot of details in the blog post but I suspect this was also foreseeable.