> it should stop talking to him altogether, and arguably Google should have moderators reporting these people to relevant authorities for wellness checks and interventions
I agree. This seems very reasonable and I would welcome regulations in this area.
The gray area imo is when local LLMs become “good enough” for your average joe to run on their laptop. Who bears responsibility then? Should Ollama (and similar tools) be banned? Where is the line drawn.