undefined

points

[-]

It's a zero day in some caching proxy who may not have had that threat model in mind. I guess its primary purpose is caching packages, not restricting internet access.

That's a far cry from finding a zero day in a hypervisor or even Docker, which OpenAI conveniently left out in their first statement by simply calling it a "sandbox".

by 0xDEAFBEAD2 hours ago|

parent|

[-]

I can't say for sure, but intuitively this type of vulnerability seems like it could be handy for an attacker trying to move through a target network? In any case, JFrog seems to be taking the situation quite seriously

https://jfrog.com/blog/jfrog-and-openai-collaboration-on-zer...

by justincormack1 hours ago|

parent|

prev|

[-]

That is the threat model, that is supposed to be safe isolated access in a vpc that isolated applications can access but it has external access.

by cmiles81 hours ago|

prev|

[-]

The deeper question though is why is someone running a model that’s supposedly so dangerous in an environment that can even get access to the internet. That just appears reckless.

There are ways to test supposedly dangerous things. What was on display here looks more amateur hour than serious testing.

by nickpsecurity2 hours ago|

prev|

[-]

I wonder if whatever had the zero day was written in a memory-safe language with strong authentication and a secure parser. Such were the recommendations to stop many 0-days before GPT-2 was invented.

If it had poor security, the attack would be both evidence of poor security and proof that the agents can compromise poor security which might still be amazing.

by justincormack1 hours ago|

parent|

[-]

Its likely written in Java, so memory safe.

by PunchyHamster3 hours ago|

prev|

[-]

In OpenAI sandbox. which was probably vibe coded. That is to say it is probably far easier to achieve than escaping something more battle tested.

> Note that regulation is useless here, because black hats don't give a crap about regulators!

I'd argue more than useless and actively harmful, as you get denied access to tools that black hats use against you.

by 0xDEAFBEAD3 hours ago|

parent|

[-]

"To gain Internet access, the models identified and exploited a previously unknown zero-day vulnerability in Artifactory, a package registry cache proxy. We disclosed this vulnerability, along with other Artifactory vulnerabilities our models identified as part of our review, to the vendor."

https://openai.com/index/hugging-face-model-evaluation-secur...

Based on some quick Googling, Artifactory was created by a software supply chain company called JFrog, which is publicly traded and valued at around $9 billion.

>I'd argue more than useless and actively harmful, as you get denied access to tools that black hats use against you.

Indeed. Still, perhaps the US and China can coordinate to stop releasing new models with even more advanced capabilities. Such coordination could be bad for big AI companies in the US, since they've been making money by offering more powerful models than the open-weight stuff which comes out of China. Any halt to the AI race will create equalization and erode their competitive edge. Speaking for myself, I personally wouldn't mind seeing US AI companies take a financial hit from such an an international agreement.