Nah, I think this is a common misunderstanding of how LLMs work, where people think that they mimic the pre-training data. Stylistically everything you see is an artifact of post-training, which is from reinforcement learning not from absorbing mass amounts of text. At some point a person or more recently a bot gave a thumbs up to an A/B tested response including em-dashes and claudisms galore.
replyYes. This completely explains sycophancy at least.
replySo question then, why is it so hard to make an ai that doesn’t do these things? And why do Claude and ChatGPT have the same -isms? They’re both doing the same a/b post training with the same decisions?
replyIt would require changing humans first.
replyYou don't blame the puddle for taking the shape of the hole.
replyThere's layers, some of token selection is fingerprinting https://github.com/google-deepmind/synthid-text
replyYeah, but I understand that fingerprinting is essentially a pseudorandom overlay onto a pseudorandom base signal. And unless you have access to both the random number generators and the weights, I don't think you can detect it?
replySo "fingerprinting" operates on a totally different and basically invisible level, as opposed to the obvious stylistic patterns that the average programmer can identify in about 2 sentences.
No, there's no reason chatbot behavior would have anything to do with frequency of text in pretraining.
reply