upvote
> Base64, Unicode substitution, emojis, output of obfuscated but "harmless" code run in a sandbox, image steganography, etc that could be endlessly disguised without a human even being able to see it, yet alone fall for it

Like a whisper or a morse code pattern or a post-it stuck in the middle of a stack of fresh printouts saying "${employee} is threatening to kill me please call 911" or...

Yes, LLMs and humans have different sensory inputs. That's immaterial; the "problem" isn't in the intersection of LLM and human sensoria, but in what happens once those inputs reach the model/brain.

> I always hear the "but humans fall for social engineering too!" line used reflexively whenever yet another prompt injection attack gets reported and it drives me crazy.

Weird, because all I see over this site is people reflexively claiming that prompt injection is like SQL injection and "lethal trifecta" is a software engineering problem.

It is not. Prompt injection is, fundamentally, the same class of thing as social engineering. Treating LLMs as regular software is a fundamental category error. Until people finally get that, we won't get much progress in this space, because all the effort gets wasted on barking up the wrong tree.

reply