undefined

points

[-]

I agree actually. We built an open source agent-agnostic sandbox that we are landing improvements to, much more to come: https://github.com/anthropics/sandbox-runtime. That said the sandbox is not perfect, and any sandbox needs holes punched otherwise it is not useful in practice (at least for inference and MCPs, and almost certainly more). We will be doing much more in this area.

by coldtea11 hours ago|

parent|

[-]

>and any sandbox needs holes punched otherwise it is not useful in practice

Don't assume our individual use cases. It can be perfectly useful with access to just one folder and zero network asside from the API.

by grey-area20 hours ago|

parent|

prev|

[-]

That’s good to hear, it’s a much better approach than expecting humans to read commands and approve them or just trust that the LLM can set the proper horizons for its world itself while working.

by Kim_Bruning1 days ago|

prev|

[-]

Making a sandbox both fully secure and fully useful are contradictory requirements. If you want the agent to do any real work, you have to leave gaps.

With SOTA agents that’s a real bind, because they’re very likely to find those gaps. Whether they exploit them, close them, or leave them alone then becomes a prompting challenge yet again.

I don't think this means don't sandbox at all. You can prevent a lot of small and medium accidents by locking down or gating access to hidden files, ssh, sudo, etc.

But your sandbox can never be the whole story.

by coldtea11 hours ago|

parent|

[-]

>Making a sandbox both fully secure and fully useful are contradictory requirements.

Any tradeoff deals with "contradictory statements". Doesn't mean we should always side with one side on a tradeoff.

Nobody said anything about "fully secure" and some arbitrarily "fully useful" combined, anyway.

For certain people it could be 100% fully useful with just access to a few fixed folders and a few fixed APIs.

Many of us don't want it to drive our lives, just do some specific this or that.

by Kim_Bruning9 hours ago|

parent|

[-]

Right! That was the standard view until fairly recently. But I'm pointing out something else.

This year we've learned that If a SOTA LLM decides that the sandbox is standing between it and its current objective: well, it has the ability, the time, and literally nothing better to do all day. Odds are, if you've configured the sandbox for anything useful, you've also provided a crack it can use to get out.

Again, doesn't make a sandbox entirely useless. Does mean you can't just put it in a really tight box and call the job done. If anything, the strict sandbox might even hurt, because it's more likely to get in the way.

(By analogy: consider the safety issues posed by a dam without a spillway.)

by aftbit6 hours ago|

parent|

[-]

This is much more of a problem if you're attempting to run at a massive scale, beyond what humans can audit. When I'm doing "artisanal" agentic programming, I'm still in the loop in the sense that I'm watching what it's doing and ready to hit Esc if it is doing something dumb. If the AI trips over the sandbox, I can cancel it and do whatever it wants me to do before it starts trying to engineer its way around.