It's still strange to me that an ability to perform arithmetic (and not just memorize a giant table of results) could emerge from training on language.
replyIn principle a large enough neural network can provably approximate any continuous function. Since arithmetic can be represented with a concrete grammar, it's not surprising to me that an LLM could contain an abstract representation if trained correctly.
reply