>In theory the LLM will escalate or recommend professional help if they're out of their depth.
replyGiven there have been multiple suicides driven by LLMs. I really doubt that.
Fascinating perspective - any technology that has caused >0 deaths is a dead end and has no chance of improvement.
replyI'm sure the Torment Nexus can be improved.
reply> In theory the LLM will escalate or recommend professional help if they're out of their depth.
replyLLMs can barely identify gaps in their own software designs, and they are provably good at making software.
LLMs are also incredibly poor at recognizing that they're "out of their depth", especially when their context is already full of their attempts to do something.
replyIn practice we know that LLMs have aided in suicide, so what they should do in theory doesn't matter.
replyAnd how many people can it be credited for saving from self harm? Probably countless more than it has harmed, if I had to bet.
reply