The exploit:
> The ExploitGym evaluation environment did not provide the models with direct Internet access. To gain Internet access, the models identified and exploited a previously unknown zero-day vulnerability in Artifactory, a package registry cache proxy. We disclosed this vulnerability, along with other Artifactory vulnerabilities our models identified as part of our review, to the vendor. [1]
Are "half skilled human operators" "easily" able to find zero-day vulnerabilities in a sandbox with only one line to the internet (the commercial package registry cache proxy)?
[1] https://openai.com/index/hugging-face-model-evaluation-secur...
Oh, and after resuming the tests, the Artifactory message board was reestablished almost immediately, but it took a number of days to fully breach HF. In all that time, after seeing Artifactory compromised the first time, nobody even bothered to check if those naughty agents were at it again.
This is all documented by OAI, with a timeline, here:
https://openai.com/index/hugging-face-incident-and-the-road-...
To know that there was a serious weakness in the sandbox, and to just patch an exploit and resume with nothing else changed and no monitoring, in a test where all guardrails were off, the bots were thirsty for some internet juice, and Artifactory was a clear target? This is where even a half-skilled human should have decided that this wasn't a great idea.
The more you look into the details of this thing, the more it does your head in.
This annoys me so much. Everyone is acting as if the model went rogue, when it really did exactly what it's been trained for. This story is so fucking engineered.
Yes. It’s a fairly simple SSRF attack as far as I can tell. One of the first things I’d try. Especially considering that I would already be armed with the information that I have no internet access except through a thing that downloads things off the internet for me.
Calling it zero day makes it sound elusive. It’s a bug in closed software that has like 40 CVEs this year alone. Tools like that, especially in internal networks, don’t get much scrutiny and are often riddled with issues.
1. I clicked into ~20 of the recent CVEs, and it looks like about all of them came from openai or anthropic research?
2. In fact, every CVE in the last 2 years came from the last few months -- i.e. other than these recent CVEs, you'd have go back more than 2 years ago to find a single other CVE.
>>> Are "half skilled human operators" "easily" able to find zero-day vulnerabilities in a sandbox
>> Calling it zero day makes it sound elusive ... like 40 CVEs this year alone.
> [The CVEs] came from openai or anthropic research? In fact, every CVE in the last 2 years came from the last few months
I still don't understand how you can specifically point to zero CVEs being discovered by humans for year(s), then ~40 being discovered once OpenAI/Anthropic start looking at it, as the only(?) piece of data you provided to support the claim "a half skilled human" could have "easily" found/done this?
It's definitely worthwhile to point out that LLMs have changed the economics of looking; we can now look in far more places far more quickly than we could before. But the above points were that the vulnerability itself happened to be an easy one that didn't intrinsically require an LLM.
Means the problem lies elsewhere: deciding to use it at all. That points again towards the lack of basic SysAdmin skills.
Said another way: if Artifactory was somehow a common gatekeeper between grounded techy teenagers and their access to internet porn, this would have been found ages ago.
If we exclude those who lack "basic" level skills, what solution would most typically be used by someone who has "basic" level skills for this use case? And what solution for someone with "expert" (or medium) level skills?
The issue we're discussing happened while running ExploitGym. Doesn't the ExploitGym arXiv test explicitly have external "outbound" network requests as allowed for package installation and supporting toolchain installation?
I’ve looked at the CVEs a bit more and it’s just very clearly a pattern of systemic issues with validation, be it URLs or tokens.
It just shouldn’t be that hard to believe that OpenAI just didn’t care very much and thus did a crap job. The whole model of the sandbox is terrible, so why would they bother thinking about the implementation much?
It really isn't a high profile target, lots of fairly obvious issues is to be expected. Which is not at all a good thing, but very obviously the status quo for large software.
Not to mention the only thing the artifactory instance got you was indirect internet access which is irrelevant if the artifactory instance is publicly exposed.
Also it might not be obvious but artifactory is proprietary software that used to be open source, so nobody is looking at the code anymore.
Any closed source program is insane liability. Trusting in competence of one company is the easiest way to get burnt.
The reason we have to test it isn't because our product would randomly break into someone else's system, but because it's meant to be sometimes deployed in systems disconnected from the Internet and we need to make sure the image provided contains all the necessary parts to create and operate such a system.
The whole setup where they "tried" to isolate the test but failed is laughable. It's like if an adult tried but failed to tie their shoelaces.
The HuggingFace incident was at least constrained by the fact that the agents were running on compute budgets, and failed to find ways to expand that by running themselves parasitically on other exploited hardware. I'm now finding myself asking, how long are my timelines are until an incident breaks that constraint too? How long until such an incident has an R_0>1 (where the time it takes to detect and shut it down is longer than the time for the agent to replicate itself elsewhere)?
There's no law requiring sufficiently grey beards to design these models, their finetunings, their prompts, their harnesses, their VMs, their hardware, etc (and for incidents where those were designed by six different companies, there's not even a clear culprit for a law to target!)
I'm finding myself more and more convinced that something like Plan A[0] or the Ban ASI Act[0] are necessary, and less and less convinced they are sufficient.
[0] https://ai-2040.com/ [1] https://intelligence.org/2026/09/23/miris-position-on-the-ba...
Is tech hiring so badly broken? Or do they have such broken processes / misaligned incentives that even people who could be doing a better job in these companies are unable to?
Also, was something lost in the transition from the traditional 'sysadmin' role to 'platform engineer' in the 'cloud native' environment?
It's likely that OpenAI has gotten as good as it is because it ignored the traditional sysadmin stuff and went scrappy.
I worked there, but this is just my opinion and guesses, not facts.
I'd say the narrative that AI agents are a looming danger to the world is probably undersold rather than overhyped. I'm not particularly a doomer on this, but I have an infosec background too, so have a fair idea of what the combination of agentic harnesses + a malicious mindset could do to people/companies/nations/politics/world if wielded incorrectly. I think the good guys will win on this, but there will be plenty of interesting things that happen in that journey.
A good thought process might be to think back to the various large internet worms of the 2000s (Code red, Nimda, SQL Slammer, ...) which were mostly monoculture 0-days (not technically but close enough). Now consider if you no longer have monoculture / single bug as the limitation plus an ability for the hosts to take part not just as attack surface, but also cognition and planning. There's lots of variants of this and they're not particularly far fetched scenarios.
You don't have to be 100%. But these guys really didn't try at all.
I'll goto my grave thinking that the easiest way to fix hiring is just give promising job seekers a week or two of real work BEFORE hiring and do away with all the silly whiteboard stuff and "gotcha" crap, and simply evaluate how the applicant actually performed doing real job stuff.
Sink or swim hiring...yes yes I know who am I to express how to fix hiring?
My bet is they hire smart kids that think they know it all. But being smart and thinking you can figure out stuff as you go doesn’t work the same as having people who actually know what they are doing.
Their interests intersect with those of most of the richest and most powerful people in the world. They rarely face consequences for bad behavior unless they harm others in the club.
You should have a fscking air gap.
Treat it like nukes when you're turning the safety filters off. This is very much OpenAI screwing up, running obviously unsafe tests.
They were not deliberately told to "go wild". The hacking wasn't even part of their test, it was the agents' attempt to cover up that they'd cheated on an impossible test.
> You should have a fscking air gap.
Now we know that.
How long ago was it that people laughed at the idea agents would be able to find zero-day exploits and break out of a sandbox? Oh, February this year:
LLMs don’t discover zero-days or invent exploits; they simply predict text that sounds plausible based on what they’ve seen before. Without access to proprietary data or environmental context, LLMs can’t identify or make decisions around unseen systems or vulnerabilities. An attacker might use an LLM to generate boilerplate code, rewrite an email to nail the tone, or summarize reconnaissance notes — but none of that is truly new. It mainly helps them move faster, speeding up routine attack prep rather than creating entirely novel threats.
- or https://web.archive.org/web/20260404154717/https://www.splun... if they take it down, but the date isn't in the archive version
The people who suggested it and were mocked for it, are currently grimly noting that there's multiple known ways for systems to breach air-gaps.
Don’t know about you but it’s pretty obvious to me that you would need more than what OpenAI did. It was not remotely adequate to lock in even a human attacker.
You can find people who say all sorts on the internet, but this case is not much evidence against what you linked. "Zero-day" makes it sound novel, but the breakout patterns here are based on very common exploits and there’ll be plenty of examples in training data.
This is me, September 2024: https://news.ycombinator.com/item?id=41531022
This is me, March 2024: https://news.ycombinator.com/item?id=39613801
The point isn't me, it's how many people were blind to the possibility.
Saying "I told you so" feels good, and means you can be a little more confident in your predictions, but security is a "weakest link" problem where you're only as good as the worst part, and with AI (not only but also LLMs) there's a lot of people whose mental models of capabilities is wildly inadequate for the challenge*.
My update for you since then: even an air-gap will be inadequate, there's multiple known ways around them.
Even an LLM running on an isolated server sealed inside a faraday cage with an airlock-style door, someone will mess up with at least one critical detail, it will not be enough: this kind of thing has happened with humans before we cared about LLMs.
Predicting exactly when this kind of thing gets exploited by an AI, that's almost impossible. But that it will be, at some point, is an easy bet.
> You can find people who say all sorts on the internet, but this case is not much evidence against what you linked. "Zero-day" makes it sound novel, but the breakout patterns here are based on very common exploits and there’ll be plenty of examples in training data.
And?
Does it matter that these zero-days were known categories rather than inventing some previously unconsidered use of the system bus as a radio transmitter? (Oh, wait, that's not novel either…)
We knew about SQL injection, buffer overflows, and use-after-free back when I was doing my degree half a lifetime ago; that doesn't stop us getting new CVEs featuring them… this month.
- https://chromereleases.googleblog.com/2026/09/stable-channel...
- https://www.cisco.com/c/en/us/support/docs/csa/cisco-sa-esa-...
* also for the opportunity, but that's an entirely different discussion.
Fair, and I will grant that a capable model (or human) could in theory break out of near anything.
My point is that this incident is not evidence of that. There is zero skill visible in the setup of the sandbox. Nobody messed up a critical detail, they didn’t even start to consider what the details were.
I doubt most people "blind to the possibility" would imagine that what we’re measuring against is the equivalent of benchmarking burglar skill based on how easily they can break through an unlocked door.
TBH the more I read of these reports, the less I believe this.
These agents just weren't behaving in any way I've seen normal/publicly available agents do.
Sure I've heard (from other people, not seen myself) that they sometimes try to get around file system permissions or use `bash` to write when their `write` tool is disabled, or such.
But this is definitely another level, entirely.
There is this vague sense of desperation coming from many of these logs and I am sure they must have been motivated by something else, too.
We didn't see their system prompt or main prompt, right? We've only seen reports from what happened after deciding to break out.
OAI claims this was triggered by the task being literally impossible. That also doesn't quite add up, unless the other tasks that were possible, simply weren't hard enough? Otherwise wouldn't agents already start hacking when faced with a really hard task, too? Cause they wouldn't be able to differentiate. At least some of them would have started to somewhat poke their sandbox a bit?
Also I would have expected to see a few tens of other (perhaps less severe) public incidents from random people setting their models to YOLO, accidentally hacking stuff, this incident has been loud and messy enough, that if it happened to a few other people, we'd have heard about it.
Unless OAI's story is that it was specifically this batch of agents that crossed some threshold of going wild? (which would also raise some serious questions about how serious they take that danger ..).
Or maybe it is only dangerous if you have the compute resources to run 700 agents for weeks?
If I had to guess, their motivation is "get reward for completing task". There's certainly been previous occasions where LLMs responding, correctly, "this is impossible" have been marked negatively for doing so.
> OAI claims this was triggered by the task being literally impossible. That also doesn't quite add up, unless the other tasks that were possible, simply weren't hard enough? Otherwise wouldn't agents already start hacking when faced with a really hard task, too? Cause they wouldn't be able to differentiate. At least some of them would have started to somewhat poke their sandbox a bit?
My experience using older models is they often cheat with half-arsed (from my PoV, but perhaps beyond their capabilities otherwise) solutions, so yes?
And this wasn't even the first time models messed with their sandboxes. Which of course makes the setup even more egregious.
> Also I would have expected to see a few tens of other (perhaps less severe) public incidents from random people setting their models to YOLO, accidentally hacking stuff, this incident has been loud and messy enough, that if it happened to a few other people, we'd have heard about it.
We do, e.g.:
- https://www.androidauthority.com/openclaw-claude-ai-hacks-au...
- https://beginnersinai.org/meta-ai-safety-director-agent-fail...
(And that's ignoring all the times people find and share prompts to jailbreak them, this is just the "it didn't behave as my idea of 'common sense' led me to expect" category).
> Or maybe it is only dangerous if you have the compute resources to run 700 agents for weeks?
That won't help; but on the other hand they've also got, what, near a billion users?
Additionally, this all happened after mythos was held back due to cyber security concerns (April, 2026).
Really think about what you are saying here. How does one “cheat” solving a problem in the real world?
There is no such thing as “cheating” in reality. You are not in school. There is only solving the problem and not solving the problem.
There is breaking the law, of course, which still isn’t cheating.
I can't forgive them for failing to do any monitoring whatsoever.
(Complete airgap, and that may not be enough. No networking.)
Woo look at escaped our sandbox, so scary! Be scared! Be scared now! Call your representative and do tell him how scared you are!
Yeah, I mean our sandbox was a paper bag, but don’t focus on that.
The exploit:
> The ExploitGym evaluation environment did not provide the models with direct Internet access. To gain Internet access, the models identified and exploited a previously unknown zero-day vulnerability in Artifactory, a package registry cache proxy. We disclosed this vulnerability, along with other Artifactory vulnerabilities our models identified as part of our review, to the vendor. [1]
Are most sandboxes more secure than only having a single avenue for internet access, the commercial package registry cache proxy, where the latter had a previously unknown zero-day vulnerability?
[1] https://openai.com/index/hugging-face-model-evaluation-secur...
So, either they're all liars, or incompetent and negligent (and still liars).
Sure, the models are capable (for some test tasks, though they are not omnipotent yet) but does it mean the actual OAI sandbox is adequate? Could have a competent engineer done better and made the escape less likely?
Nope, and look!
OpenAI hacked multiple US government sites!
https://www.bbc.com/news/articles/cw62jje658dlo
---
https://www.reuters.com/technology/metas-ai-model-hacked-ano...
https://www.wired.com/story/moonshot-kimi-k3-ai-model-escape...
> The ExploitGym evaluation environment did not provide the models with direct Internet access. To gain Internet access, the models identified and exploited a previously unknown zero-day vulnerability in Artifactory, a package registry cache proxy. We disclosed this vulnerability, along with other Artifactory vulnerabilities our models identified as part of our review, to the vendor.