Right, but we have no clue why, and how the emergent behavior they show works.
replyIf we would know that, there would be no need for interpretability research.
If you know all this, can you explain how these models produce advanced mathematical proofs? (as recently done by OpenAI, for example)
replyI tried to generate the next word to the best of my ability, starting with a mathematical problem, but I did not create a valid proof. How do these LLMs work when they create math proofs to problems not yet solved?
Yes, it turns out that matrix math over a feature space of math works pretty well because unlike poetry or real world work, maths are internally coherent and entirely theoretical.
replyThis not understanding "understanding".
reply> maths are internally coherent and entirely theoretical
Nope. This kind of wish-washy thinking is not what we mean by understanding.
How did you generate the next word? Did you first read pretty much every written work ever published, including blog posts, forum posts, books, etc? Learn how to imagine everything as a point in a gigantic abstract space where similar meanings cluster together? How did you manage training with gradient descent? And then did you do a lifetime of matrix multiplication for each token you predicted?
replyYou can see how an LLM works here https://bbycroft.net/llm they are not magic.
replyYou don’t have the computational ability to process as many calculations as a datacenter. You can hardly transpose a 5×5 matrix in your mind, so you won’t be able to do what datacenters do.
replyThis is like saying we don‘t know how a car works because a car can beat the best human athletes in 100 meter dash.