>As for chess, although an LLM knows the rules of chess, it is not expected to have been trained on many optimal chess games

Well, it's read every book ever written on chess so you would expect it to be at least half-way decent.