The exception is translation. Translation is what transformers were originally developed for. LLMs shine in translation, and creating code is, after all, a translation from natural language into a programming language.
replyA lot of what current LLMs are good at seems to boil down to translation:
* Translate some prompt into a planning list of individual TODOs
* Treat each TODO as a new translation (e.g. from TODO to code), or call some external tool (lookup something on the internet, static code analysis, database request)
* Translate the result(s) of these TODOs into a final text, or into a new TODO list
To me, this is interesting, because maybe the Homo Sapiens intelligence simply developed as a side effect of communication (translating words into actions).
You could take that a step further and say everything is information, and our brains transform it into reality (Donald Hoffman).
replyI think that's where it goes, yes. The ability to model the world internally predates spoken language. We (and other animals) already translate what we _sense_ into that internal model. Language is just another translation; all communication is bidirectional translation, internal modeling/thought is wordless.
reply