upvote
Agreed and this is a very real thing to be concerned about.

I like the idea of leaning on a decision like Jev model to identify such things.

reply
The only attacks possible would be the tools given to the agent.
reply
Assuming it can't perform a sandbox escape, at any rate. Maybe a Jev-like decision model would be a better idea.
reply
Or just don't allow tools, write a simple python script that uses the API and doesn't do tool calls, etc. etc. etc.

Table stakes for being an engineer these days should include knowing how to control LLMs, it's as fundamental as a working knowledge of databases. One doesn't need to know how design a data model from scratch or code up that python script using the OpenAI API from memory, but should know the general shape of what's possible and how to look it up.

reply