Top bots are in the ballpark of 50kloc. Even large LLMs are not so large that they can memorize all the world's data like that. And all the fine-tuning that compression loses comprises most of the work in the first place.