In pre-ChatGPT days, listening to discussions of "AI X-risk" and "boxing", I used to think that it should be easy to just ignore arguments presented by the AI on principle, and let it out of the box.

It turns out that tons of people will tear open the box before the AI has even output anything, not despite its fearsome power but because of it.

So I really hope I'm right that recursive self-improvement doesn't work the way the doomers think it does.

The "s" in "AI agent" is for "security"