You ever wonder why recent claude models speak in riddles? I dunno, maybe all those "rare" books? They may have been rare for a reason.

LLMs get a lot of finetuning, but I suspect there are two things that can cause this kind of writing:

Firstly, some parts of the RLHF involve human graders on the LLM's performance. I suspect their general bias towards a punchy, persuasive writing style could come from what biases the graders towards preferring that response, especially in shorter segments and when the grader is not focused on writing style

Secondly, later parts of the finetuning involve reinforcement learning on achieving certain tasks which are automatically graded: stuff like coding tasks. I think this can create a kind of feedback loop where the style drifts further, and you get the kind of LLM tics which are even more extreme (it might be that they incidentally help somehow with the actual tasks, or it might be a drift that comes from the grader also now being an LLM or some of this finetuning happening on output from other models). The more recent claude models seem to suffer from this a lot, moreso than earlier ones.

A bigger question I'm interested in is why do LLMs speak like that in the first place? Is that really what you get if you took the average of the English language? It would be difficult for me to believe that.

Is there something about tuning for desirable qualities that forces LLMs to have this voice?

They are certainly trying to