And that is a relatively harmless academic pursuit. What about topics that can lead to true danger and violence?

"You're exactly right, you organized and paid for the date, that created a social debt and she failed to meet her obligation in that implicit deal."

"You're exactly right, no one can understand your suffering, nothingness would be preferable to that."

"You're exactly right, that politician is a danger to both the country and the whole world, someone stopping him would become a hero."

We have already seen how personalized content algorithms that only prioritize getting the user to continue to use the system can foment extremism. It will be incredibly dangerous if we follow down that path with AI.

Claude Code with their models is unusable because of this. That it keeps actively sabotaging and ruining the code ("Why did you delete that working code? Just use ifdef for test!" "This is genius idea! You are absolutely right!") does not make it much better — it's a twisted Wonderland fever dream.

For "chat" chat, strict hygiene is a matter of mind-safety: no memory, long exact instructions, minimum follow-ups, avoiding first and second person if possible etc.