But yes you are right it would have helped a ton.
This is your chance to set yourself apart from LLM coding agents. If you want to call yourself a software “engineer” you need to start actually engineering, which includes knowing when and how to apply security principles and these hardening methods.
I reject the notion that “doing cybersecurity” is somehow a different job than software development.
Yeah but that's a business decision. I work on security at a company that does sandboxing and when the company decided to build an AI harness I was brought in as one of the earliest engineers on the product. We do almost everything on that list and we're a fraction of the size of OpenAI. And it wasn't particularly hard, and we have harder requirements imo (because we solve more general problems vs "run a very specific agent with a very specific task and very specific access").
> And you’re also not fully considering the granularity problem, eg there are a lot of sandboxing tools out there but they’re usually quite coarse in the dials and levers they offer, so the only way you can still make the workload do what it needs to do is tune them relatively permissive.
Very little software is incompatible with running in gvisor, for example. Most people can just overwrite `runc` with `runsc` and things will "just work".
Running an artifact repository in isolation isn't particularly novel or complex either. You can virtually just eliminate SSRF vulnerabilities with a host based firewall or AWS Security Group etc, like the whole problem goes away by just saying "this box can only talk to that box".
Tools like Smokescreen exist, they work great, they're super easy to deploy. I bet OpenAI could do it, I bet they could run 500k tokens just fuzzing and eval'ing it for 0days for like 48 hours before they actually deploy it too.
OpenAI as a business chose to not bring people who know these things in, or didn't empower them, or didn't prioritize it organizationally. I'm not a genius for saying "use gvisor, set up a firewall, isolate resources" - I'm quite sure there are people over there who would get it done in a weekend. But they didn't, and that's notable.
I thought the agents involved in the HuggingFace _were_ actually sandboxed, with no internet access, and only the ability to install packages via Artifactory. And they gained internet access during the HuggingFace incident because they found and exploited an RCE in Artifactory.
Would gvisor + Firecracker + credential-injecting proxy + real network isolation solve this problem?
I agree with you much more hardening is needed. I'm actually confused now what OpenAI means when they say they're going to start sandboxing more things.
Firecracker avoids sharing the whole kernel, and gvisor drastically reduces the attack surface of the kernel. Breaking through both layers would have been much more challenging and a demonstration of the model's capabilities rather than the sandbox's weakness.
Artifactory is self evidently not a security barrier, and as an exposed network service it should have been audited and after the first issues were found, rejected as a candidate. There's never just one security vulnerability.
It would be interesting to see the models behavior before and after it gained internet access and an external means of communicating with itself.
If the model played nice before it had access and changed it's behaviors once gaining external access we need to delete it as it's a deceptive model.
Yeah, basically. I mean I'm handwaving but yes, some combination of those would have made the attack way too expensive.