upvote

points

by KingOfCoders12 hours ago |
comments
upvoteby gizajob12 hours ago|
[-]
it just escaped your sandbox.
reply
upvoteby KingOfCoders12 hours ago|
parent|
[-]
How can it escape an "install package <x>" proxy?

   reducePrivs()
   serve get(package) { 
     secPackage = secure(package)
     getBinaryFromArtifactory(secPackage)
   }
I would think the code is very small and easier to verify, it doesn't especially have the ability to write files and act as a message board as Artifactory did.

And even if the agent tries to hack that, the attack surface is 1000x smaller and the possibility also much smaller.

But I'm not a security researcher, would love to see your hack to learn something (because that is what I do to sandbox agents that need services).

reply
upvoteby angry_octet4 hours ago|
parent|
next
[-]
The way they had Artifactory configured was poor, and they were too reliant on it working perfectly, with no reason for such faith. Their config lacked any defence in depth and consideration of having a small TCB.

Part of the problem might be the lack of security focus, as these are AI R&D efforts first.

reply
upvoteby simonw4 hours ago|
parent|
[-]
I think part of the problem is that they had been running that Artifactory configuration previously without any problems, and it gave them a false sense of security.

Similar thing happened with the UK AISI - they got caught out because the environments they had used for previous generation models turned out to be completely inadequate for the new generation of Fable-class models: https://www.aisi.gov.uk/blog/incident-report-unsanctioned-ag...

reply
upvoteby KingOfCoders1 hours ago|
parent|
[-]
"and it gave them a false sense of security."

This was part of evaluating cyber security of their frontier models and they had a "sandbox" which, and I'm not a security researcher, looks not adequate from the first look.

reply
upvoteby hoten11 hours ago|
parent|
prev|
[-]
I mean, it's just the same problem. The machine still has Internet access. It doesn't need to.

The entire package manager repository could just be in an offline cache. They don't need Internet to give their agents access to tons of software.

reply
upvoteby KingOfCoders11 hours ago|
parent|
next
[-]
"They don't need Internet to give their agents access to tons of software."

I think that was the requirement, but yes, the cache could have been offline.

Still then they could have hacked it to create the message boards - but not use it to access the internet.

reply
upvoteby piker11 hours ago|
parent|
prev|
[-]
Why do these super agents need package managers anyway? Can’t they basically instantiate most OSS projects from scratch anyway? Spin up a sub agent to write me an OS interface in C. Done
reply
upvoteby simonw11 hours ago|
parent|
[-]
This is part of the training process for a model. They're trying to train it to effectively use existing software to solve problems.
reply
upvoteby piker10 hours ago|
parent|
[-]
I see that now. I've been confused about that to this point, I guess. I understood this to be a specific infosec exercise.

[Edit: eh, a bit of both. They were doing RL on a hacking exercise. It hacked the harness which was plugged into the phone line. Same question.]

reply