Yes there are gains from breaking it up and rephrasing it and simplifying it for humans to understand, but it used to be that a thing was understood first by the leading experts in the field and eventually trickled down to being common knowledge.
Now we are entering a period where things are understood first by AI systems and have to filter down to the world's leading experts.
Isn't this always the case? Maybe new to math as a largely non-empirical project, but we had knowledge about the advancement of the perihelion of Mercury for 60 years before General Relativity made it "intelligible".
LLMs know the answers to questions we were already aware of, and we can tell they're right, but we can't understand their working. That's what's new.
If a team of 100 top-flight number theorists working together had used much less sophisticated LLMs + Lean to produce this proof, we’d be just where we are today.
If that's not the standard for understanding something, what is?
I think of it like a spaghetti codebase with a giant densely connected graph of interactions. Today’s LLMs can easily “understand” and answer questions about it, but only by way of fitting all of the complexity into context at once.
The more useful form of understanding is being able to compress that giant graph down into a far smaller set of independent subgraphs. To factor it into simple, obvious, reusable building blocks, and use those abstractions to express the problem itself in a much simpler way.
That’s an instrumental definition, but it captures the idea that understanding has to have causal, conceptual linkages to a wider body of knowledge. Imagine a magical, infinite catalog of true vs. untrue statements. Every bit of knowledge is there, but we wouldn’t say it has understanding & it couldn’t teach us anything beyond its (impressive!) raw content.
This just stopped being true.
And even in mathematics, incomprehensible but valid proofs have been a thing since the four-color theorem decades ago.
It's just that no single human understands the whole. But we had proofs like that before.
Btw, see also the infamous proof of the 4 colour theorem, which faced similar hurdles.
This hypothesis was previously not proven at all. That's strictly worse.
Now it is understood by our systems.
Soon it will be by us. Our systems gave us a new and working salient.
We're building machines that help us understand faster.
There have been plenty of times in history we didn't completely understand something yet used partial knowledge to make advances. This isn't even that bad.
I’d call that a contradiction. If it’s not intelligible, then it’s not knowledge.
If something were intelligible to adults but not to children would that make it not knowledge? What's the difference if it's LLMs and humans instead of adults and children?
That’s a separate question with its own term: common knowledge. I am willing to set a much lower bar: something is known to humanity if it’s known by at least one human.
Personally, I don’t believe that AIs know anything, as they’ve been constructed thus far. Knowledge is justified true belief, and AIs don’t have beliefs, they only have utterances.
Why do I think AIs don’t have beliefs? Because they contradict themselves all the time.
But it really doesn’t matter if you think AIs have knowledge or not. If they cannot explain it to us then it’s inaccessible to us, so it’s a “tree falls in the forest with no one around” situation.
If it's formally verified and we assume the kernel is good and problem definition properly vetted, then I know the GRH is true regardless of if I know why it is. Because those too are different pieces of knowledge.
There is no explanation here, there is a GB of Lean code and it’s not like it all fits in the context at once.
Let's not sell humanity short just yet.
I agree with you about the current situation, and the near future. But the trajectory could still be towards humans pretty much as bystanders. I'm pretty sure the 'minds' in the Culture universe will be happy to indulge some curious humans and explain mathematical proofs to them.
Even more than that, in Culture there are a small number of human individuals, called a Referer, that are consulted by the Minds because they also posses similar problem solving abilities of the Minds. Optimism abounds.
As a software engineer, you can read this code instantly:
temp = a;
a = b;
b = temp;
But when it happens in the middle of a big chunk of code it still takes a little more work to read than swap(a, b);
whether you want to think about it as human attention and working memory, or token budgets and context windows, humans and LLMs all benefit from introducing abstractions that let us reuse existing solutions.AI are not very good at coming up with questions, yet. They have no curiosity, no goals, no drive. Unless someone tells them to do something, they don't do anything.