Humans are vulnerable to scams, cons, phishing and social engineering. This is a vulnerability whose root cause we not know how to fix. We can only alleviate its effects by re-engineering the environment around those vulnerable humans.

LLMs are just as gullible, we just tend to call their vulnerabilities "prompt injection", even though it's basically the same class of attack under the hood. Unlike humans, though, LLMs are a lot less diverse (you and I are using the same Opus 5.5, but we have different brains). This makes attacks on LLMs much more predictable and easier to pull off, just as a phishing campaign that targets one specific person is easier than an attack that scales to millions of diverse victims.

In the age of agents, phishing will continue to be the biggest attack vector of all, it's just the victims that will change.

Well, there's definitively a pareto frontier of freedom and avoiding bad stuff. And many companies ain't on it.

(In practice, I found Google to be better at having (internal) security controls that are reliably but also don't get in the way than eg banks I worked at.)

the difference being, that if you build those kind of approval processes up with AI agents, the system will function a bit faster. you could have a web reading agent. a web writing agent. a coding agent. a credentials agent. etc. and have everyone make these formal requests.

it will catch quite some prompt injections, as the prompt injection has to promptinject a secondary system in order to be successful.

> the more bad stuff happens (and the more consultants you allow in)

In your parenthetical, you just said the same thing using different words