upvote
This feels fundamentally right to me.

The best compression you're going to achieve is when you best understand(i.e. encode) the truth of things.

Perhaps you're thinking of that too narrowly - it is not that the model gets good at understanding how a bridge stays upright; it has to also understand the truth of how a specific person might understand it, and how that specific person might put it into words.

Do that right, as slight deviation of some understanding of how humans respond in general instead of encoding for a single person, etc, etc, etc - and you reach optimal compression.

It's more efficient for the model to encode _how_ an idea is wrong, than it is to encode the wrong idea.

Its how/why they detect when they're being evaluated.

reply
I think the perplexity is measured on the pre-train checkpoints.
reply
To effectively pretend to be a fool, the model needs to be smarter than the fool. That's why perplexity decreases.

The meta game in pretraining is it correctly assumes the context/mind state of the writer of whatever material under test. To do this effectively requires further modeling and understanding, which is a strong predictor of capability and is a much more difficult task than simply being bad at it.

reply