A human being has general intelligence and needs A LOT of training and finetuning to become good in chess.
And there is a relevant and significant difference between the expectation of an AGI and an ASI system.
Your assumptions/intuition about generic human intelligence feels quite incorrect, considering LLMs currently play better than a brand new human player would (presumably without any attempt to fine tune them specific on chess, such as playing thousands of games).
They’ve ingested all the literature on playing chess, a brand new human player has not.
We seem to be moving goalposts to the point that humans don’t even live up to the expectations of the AI critics. The only way you get better at chess is by playing a lot of games and learning from mistakes, that goes for humans or AI agents, not simply by reading about chess.
How can you play without being aware of the rules and how can you learn from your mistakes without knowing they are mistakes? That’s what I said about reading a book of two. It is to kickstart the process. Then mastery is gained over time through practice.
This kickstarting then gradual refinement is how most people learn. And the foundational knowledge stays. Even a basic player knows to not do illegal moves.
It feels like you're trying to say that humans never make illegal moves while learning chess, which doesn't match with my experience. I'm trying to understand your overall point.
You have the first stage, pre-training, which is learning from next token prediction. That's where the model memorises a lot of facts about things and generally gets good at forms of writing. It's like reading a lot of books on programming and reading through a lot of source code. It's learning how to autocomplete code, essentially. Doing that requires a developing a reasonable understanding of code, but it's also learning how to autocomplete bad code as well as good, and won't make it a "good" programmer.
Pre-training uses a method called Cross-Entropy Loss to update the weights of the network.
Then comes post-training. This is where the model is trained against huge sets of example problems, like fixing a bug, adding a new feature based on a spec, etc. They are set the task and try to complete it inside a training environment. Once they're done, their complete solution is evaluated (either by humans, or by some separate evaluation model that was developed based on human feedback) and they are updated based on whether the solution was good or not.
Post-training uses a different method called Proximal policy optimization to update the weights of the network.
So these really are very different forms of learning, and mainstream LLMs are not post-trained to be good at chess. They could be. You could easily create a reinforcement learning environment that evaluated and improved their ability to play and win at chess. The result would be a very strong chess playing AI, something we know is possible because the strongest chess playing programs we have are neural network based, but it is not a priority for AI companies.
Coding is a matter of translating the natural language description of a problem to the code specification while keeping the semantics fixed (and imputing the unspecified semantics as necessary). It is not considerably more difficult than translating between two dissimilar natural languages. Chess isn't a matter of language translation, but a compute heavy game of finding the best move out of many possibilities with wide variation in the quality of each move. Chess takes directed practice and reinforcement whereas language translation does not.
People think that if one mention exists in the training set, then the LLM is perfect at it.