I have a personal theory: LLMs are *fundamentally* handicapped at perceiving what's going on in the mind of the human (this can't be "innovated away") and that's at the root of what makes them suck at conversation.
Next time you're chatting with someone, notice how much understanding is shared without anything being said. E.g. the other person might share something deeply disappointing, and they can tell without you even saying anything whether you get what they're going through. This unspoken-yet-communicated information guides the conversation. Or as another example: humans can read the room -- you walk into a room and immediately adjust your demeanor based on what you see and sense.
LLMs are totally blind to things like this, and this adds an inescapable awkwardness to interacting with them. I don't believe they'll ever grow out of this. Which thankfully implies more long term demand for humans instead of robots. :)
Does anyone have a read on if this is primarily a Claude issue, or if all LLMs do this?
I think this is one reason why LLM text is pretty exhausting to read for long stretches.
It's possible that this quality you describe stems from the extensive training corpora utilized by the major AI labs. These almost certainly include work from the esteemed economist Jacob Silj: