upvote
I would dispute that we completely understand how LLMs do what they do. An analogy would be that we meet an alien race that gives us their proof of the Riemann hypothesis translated to Lean. Plus of all other provable mathematical conjectures we come up with. And we might kind-of understand how those aliens do math. It would still be a bit sad that we'd be relegated to consuming what the aliens give us, and never be able to ascend to the level of the alien mathematicians
reply
We didn't meet a hypothetical alien race though. We created it. We are giving birth to something that will soon surpass us in every conceivable way.
reply
That doesn’t make much of a difference. It’s alien, and for the most part we didn’t design it to do the things it turned out to be able to do by happenstance. We might as well have grown the aliens in a test tube by some unexpected gene manipulation.
reply
Over half the cells in our bodies are not part of us. The lines we draw are seemingly never respected by nature.
reply
An analogy here is evolution. We understand what it does on a basic level. There is natural selection and species that are well adapted to their environment thrive.

This absolutely wouldn't allow us to predict what species will emerge and that something like humans would be the dominant species.

Similarly, we understand the mechanics of how LLMs work, but that does not make it predictible in any way.

reply
We definitely don't understand how Claude does what it does! We know how to grow a Claude, but we don't understand what goes on inside a Claude.
reply
I remember when about 15 years ago, I wanted to make my neural network home work based on proven math. I had to train a model to learn basic number recognition. The math said 100000 neuron is what's needed, so I typed in that, and my laptop immediately froze completely, and died a little later. I learned on that day that we know empirically that a few 10s are enough.

We have the same thing with LLMs. We don't know how they work. We don't understand them. We have only empirical, statistical indications, nothing is proven. We know how the small parts behave, we really understand those, and also the different algorithms. But they whys? No. We're getting better, but still far away.

reply
If we don't know how LLMs work, how did we create them? They didn't just spawn suddenly on someone's hard drive.
reply
What does 'know' mean? And what does 'mean' mean?
reply
Trial and error. A ton of trial and error, and a lot of guesses. More wrong than good guesses.

Basically every science field is based on this.

reply
And presumably mathematicians understand the output, the proof, whatever it is that came out of the process. I don't feel like humans have to be the one doing all the manual math and figuring out all combinations.
reply
I mean I understand how cars work but that doesn't mean I'd feel the same accomplishment of driving 42 km as running a marathon.
reply