It depends on what the tasks are. Text-native work like programming or interacting with a terminal should be a lot more immediate for LLMs than something abstract like Starcraft, DOTA, Minecraft or Mindustry. LLMs certainly know the rules for these games, but processing game state and formulating a plan in the form of a Python script or MCP call is a rough debugging loop.

Minecraft is really the perfect example, because humans get all sorts of unfair advantages. Even a young player has enough spatial reasoning to recognize and remember landmarks, visualize large projects or explore without getting lost. These are simple instincts that LLMs will struggle to match even with huge context sizes and native multimodal input.

I dunno. Aren't we somewhat similar? Why do we use software?

> Mining ore in Minecraft shouldn't be such a challenge for LLMs and yet apparently it is!

It seems like it wasn't really a challenge at first, but GPT has a major anxiety disorder and immediately retreats at the sign of any adversity (in this case, dying) and tries to be as safe as possible.