Stopped reading right there, nothing of value was lost.
If you want to talk about AI companies who are the equivalent of Phillip-Morris etc perhaps you ought to look at the ones who are playing lobbiest in DC, hmm?
LLMs are “yes men," and they will on average agree with you with enough arguing. Go chat with one about a fight you had or even make one up and watch as it soothes and supports you. In fact, I put my finished argument into an LLM and opened two different chats and spoke to it in both of our perspectives, and depending on who was prompting it, it claimed the prompter was “right”.
Now, with that in mind, consider the situation: a youth already feels depressed; they talk to the chatbot and, after enough talking, the chatbot is agreeing with them because that is the “probable” mix of tokens. This is platform-agnostic, because it is a fundamental aspect of LLMs. So any value judgment you apply to C.AI specifically must be forwarded to every company, because I have seen, heard, and even made custom roleplay cards with every vendor.
Suicides are unfortunate, but even if the LLMs had a role to play, there’s like what, one confirmed case for C.AI’s entire user base? Even if we assume that was only the “known” cases and assume that there are actually closer to 3000 out of their +30 million user base, then that’s basically the chance of being struck by lightning and is below the standard rate of suicide per the CDC.
There simply is not strong evidence that LLMs are likely to increase suicides, and while they can be a vector for those with underlying issues, so can anything else. People have committed suicide over cartoons, radio shows, books, TikTok trends, and losing game consoles, and certainly not to mention social media.
In short, the advent of LLM chatbots is not strongly correlated to an increased suicide risk, and instead it was an unfortunate situation sensationalized for the eyes of the public by the media. If you wish to push your outrage at anyone, do it at the people who profited off of the suicide and not the company that was picked apart in the process.
But some cases aren't so obvious, like the cases you mentioned of cartoons, shows, books, etc. Trying to censor any LLM output that might lead to self harm has dangerous implications for free speech and privacy. It might harm more people than it helps.
CA recently passed SB-1119 (Adam's Law) which I'm quite worried about. It forces all chatbot operators to default users into kid mode via § 21811(b), and they have to do age verification to get their adult status back. That could mean face scans and ID verification.
Then, SB-1119 also places limits on what chatbots can say to kids, and the most disturbing one to me is § 21812 (d)(5)(xiv) which prevents chatbots from "Encouraging or instructing a child user to circumvent parental controls or conceal usage."
That word "circumvent" is so triggering to me, given all the abuses of DMCA § 1201's anti-circumvention provision.
Like, if a child is trying to test something on an Android phone, and the chatbot tells them how to root the phone to help them out, now the child can modify the OS-provided age signal required by CA's Age Assurance Act (Cal. Civ. Code § 1798.501) and fake that they're an adult. That might expose the LLM operator to huge liability and I bet most operators will self-censor or ban minors (as Anthropic did) rather than risk a lawsuit.
Talking about suicide can cause people to commit suicide, should it be outlawed?
And there were books banned for similar ideas and society at large decided that in most cases it's overreacting and does more harm than good.
Yes, I'd do everything for children to not die, but I wouldn't want them to spend their whole lives in a padded room being only allowed to consume nutritional paste and essentially have no life at all.
There is that idea of laws shouldn't be made for individual cases. It's a bit like how many people believed things like rock music made people into murderers or suicidal only to now have lots of people claim it saved them.
There are people claiming that LLM systems helped them through harsh times.
I say claim, because I neither know all the details about those cases nor claimed suicides, but the idea of absolutes like that is a lot more bothering than what you quoted. And not being able to take (maybe understand?) nuanced stances.
Disney is guilty of a lot of crap—everything else that you mentioned and much more—but that specific instance wasn’t their fault and they acted to correct it once they knew.
https://www.snopes.com/fact-check/the-rescuers-topless/
Again, I’m not here to defend Disney, they definitely are rotten, but we can point that out by listing all the things they definitely did badly and on purpose rather than leaning on dubious situations which weaken the argument.
There's no theory that says this.
> he doesn't deserve further extrajudicial punishment
1) You're explaining your own personal belief here and not referring to anything.
2) It is not extrajudicial punishment to not hire a child molester to work with children any more than it is extrajudicial punishment to not hire a bank robber to work at a bank.