upvote
"The only solution is to lock every LLM query in the entire stack behind the same deterministic role-based access controls that determine resources available to the current user."

Exactly. The sooner people stop trying to replace code with LLMs, the better. The technology is fundamentally untrustworthy, and given that we do not understand it, impossible to secure.

Only extremely simple code audited by multiple human authors, with actual proof of functionality (not just testing) can be considered secure.

reply
Yeah, an agent should run with permissions no greater than that of the user on whose behalf it is executing, and ideally with less permissions. This is the scenario that is easier to fix, simply give the agent an API token with rights no greater than the user it is acting on behalf of. This could be a literal token for their account, or a limit-rights-to field or whatever, multiple possible approaches.

The harder problem is outside actors trying to prompt inject to get the agent to do something the user has rights to do but which the user doesn't want to happen. That is the hard scenario to fix, due to the nature of LLMs.

reply
Exactly!

Attempting to handle prompt injections by prompting the model (not to leak sensitive data), is like attempting to stop a fire by burning the area around it

reply
reply
Haha, nice. TIL.

So all we need is ‘controlled prompting’ to handle prompt injections :-)

reply