Yeah I think this concerns me in two fronts and this specific tendency may be the line in the sand separating LLMs from AGI.
replyOn the getting work done front, if what you’re trying to do is remotely subjective, than you really need to be sure you’re asking it the right thing and not expecting it to correct you and provide capital T truth.
On the social element. Like wow using it as a therapist or a mentor or to bounce ideas off of. What a huge trap if you expect it to correct you with a semblence of objective reality.
> What a huge trap if you expect it to correct you with a semblence of objective reality.
replyYes, as a few cases have shown, people can go off the deep end, and the LLM goes right there along with them.
The LLM has no understanding of objective reality. It's even worse than any one of the blind men trying to describe an elephant, because it has no true experience of either the thing, or the other thing it is trying to compare it to.