upvote
They never say that their hold-out dataset to evaluate perplexity is just in English. Or written text for that matter (could also include code, ...)
reply
When your performance is still worse than human of course any improvement makes it more human readable. But if you are in the regime beyond that... it'll head toward being more "oracle-like" and less "human-like" almost by definition.
reply