upvote
The difference nowadays is that the frontier of knowledge is no longer intelligible to any human being.

Yes there are gains from breaking it up and rephrasing it and simplifying it for humans to understand, but it used to be that a thing was understood first by the leading experts in the field and eventually trickled down to being common knowledge.

Now we are entering a period where things are understood first by AI systems and have to filter down to the world's leading experts.

reply
The difference nowadays is that the frontier of knowledge is no longer intelligible to any human being.

Isn't this always the case? Maybe new to math as a largely non-empirical project, but we had knowledge about the advancement of the perihelion of Mercury for 60 years before General Relativity made it "intelligible".

reply
Your example is the difference between being aware of a question and knowing its answer.

LLMs know the answers to questions we were already aware of, and we can tell they're right, but we can't understand their working. That's what's new.

reply
In this case, Claude has told us everything it “knows” (so far) and it’s an unintelligible (for now) Lean proof. I don’t believe it has some latent understanding at a higher conceptual level that’s remaining obscure.

If a team of 100 top-flight number theorists working together had used much less sophisticated LLMs + Lean to produce this proof, we’d be just where we are today.

reply
Well in the Navier-Stokes example the initial proof was completely unintelligible to humans, but other LLMs understood it and were able to answer questions about it.

If that's not the standard for understanding something, what is?

reply
Perhaps it could be considered understanding, but it’s an unsatisfying brute force sort of understanding.

I think of it like a spaghetti codebase with a giant densely connected graph of interactions. Today’s LLMs can easily “understand” and answer questions about it, but only by way of fitting all of the complexity into context at once.

The more useful form of understanding is being able to compress that giant graph down into a far smaller set of independent subgraphs. To factor it into simple, obvious, reusable building blocks, and use those abstractions to express the problem itself in a much simpler way.

reply
Being able to teach it to someone less knowledgeable.

That’s an instrumental definition, but it captures the idea that understanding has to have causal, conceptual linkages to a wider body of knowledge. Imagine a magical, infinite catalog of true vs. untrue statements. Every bit of knowledge is there, but we wouldn’t say it has understanding & it couldn’t teach us anything beyond its (impressive!) raw content.

reply
And no human could understand every frontier. There's simply too much information.
reply
For every statement that humanity as a whole knows for certain that it's true, there exists a human who understands how to prove it.

This just stopped being true.

reply
That kind of certainty vs. understanding is limited to mathematics, of course we’re almost entirely certain about many things out in the world we don’t fully understand.

And even in mathematics, incomprehensible but valid proofs have been a thing since the four-color theorem decades ago.

reply
That's not true. You can take any single piece of that lean code, and it tells you exactly how to prove it.

It's just that no single human understands the whole. But we had proofs like that before.

Btw, see also the infamous proof of the 4 colour theorem, which faced similar hurdles.

reply
This feels like the wrong framing.

This hypothesis was previously not proven at all. That's strictly worse.

Now it is understood by our systems.

Soon it will be by us. Our systems gave us a new and working salient.

We're building machines that help us understand faster.

There have been plenty of times in history we didn't completely understand something yet used partial knowledge to make advances. This isn't even that bad.

reply
The difference nowadays is that the frontier of knowledge is no longer intelligible to any human being.

I’d call that a contradiction. If it’s not intelligible, then it’s not knowledge.

reply
It's intelligible to LLMs, just not to humans.

If something were intelligible to adults but not to children would that make it not knowledge? What's the difference if it's LLMs and humans instead of adults and children?

reply
If something were intelligible to adults but not to children would that make it not knowledge?

That’s a separate question with its own term: common knowledge. I am willing to set a much lower bar: something is known to humanity if it’s known by at least one human.

Personally, I don’t believe that AIs know anything, as they’ve been constructed thus far. Knowledge is justified true belief, and AIs don’t have beliefs, they only have utterances.

Why do I think AIs don’t have beliefs? Because they contradict themselves all the time.

But it really doesn’t matter if you think AIs have knowledge or not. If they cannot explain it to us then it’s inaccessible to us, so it’s a “tree falls in the forest with no one around” situation.

reply
If I see a red apple, then I know the apple is red regardless of if I know exactly how the physics of light work. Those are two different pieces of knowledge.

If it's formally verified and we assume the kernel is good and problem definition properly vetted, then I know the GRH is true regardless of if I know why it is. Because those too are different pieces of knowledge.

reply
If it were intelligible to LLMs they could explain it in human terms, but they can’t.

There is no explanation here, there is a GB of Lean code and it’s not like it all fits in the context at once.

reply
Someone else has posted a Gemini-compiled 43 page human-readable explanation elsewhere. It very quickly exceeded my own math abilities, unfortunately.
reply
Why do you think they can't explain it? Try asking them questions, they will explain it.
reply
If we know how it resolved with certainty, it's still knowledge.
reply
deleted
reply
Would this imply Wolfram's ANKOS is coming true, given it was based on previous work + his own?
reply
Hmm does Claude actually “understand” it though ?
reply
I don't think that's true at all. This proof was created by a human driving Claude. Claude itself is a human tool. Lean is a tool made by people.

Let's not sell humanity short just yet.

reply
You and the comment you replied to could very well both be true.

I agree with you about the current situation, and the near future. But the trajectory could still be towards humans pretty much as bystanders. I'm pretty sure the 'minds' in the Culture universe will be happy to indulge some curious humans and explain mathematical proofs to them.

reply
>I'm pretty sure the 'minds' in the Culture universe

Even more than that, in Culture there are a small number of human individuals, called a Referer, that are consulted by the Minds because they also posses similar problem solving abilities of the Minds. Optimism abounds.

reply
What value is there in making these human-understandable? I don't think much of math or computer science is going to remain within human grasp for much longer, regardless of our attempts.
reply
Because the same process that we use to make them human-understandable is the process we use to make them generalizable and reusable, and those same features are good whether it's humans or AI doing the work.

As a software engineer, you can read this code instantly:

    temp = a;
    a = b;
    b = temp;
But when it happens in the middle of a big chunk of code it still takes a little more work to read than

    swap(a, b);
whether you want to think about it as human attention and working memory, or token budgets and context windows, humans and LLMs all benefit from introducing abstractions that let us reuse existing solutions.
reply
In order to come up with new questions (and subsequently answers, human or otherwise) that are interesting to humans. You’ll notice that these recent proofs have all been about big, famous, and novel human questions.
reply
I don't suppose we'll write the new questions any more than we'll write the new answers.
reply
There might be no new questions then, which would be pretty boring.

AI are not very good at coming up with questions, yet. They have no curiosity, no goals, no drive. Unless someone tells them to do something, they don't do anything.

reply
This process has already started, AI supported, e.g. here: https://andrewodesky.com/rh/
reply
Have you noticed use of multiple provers in parallel? Cog, LEAN, Rocg? They all can explore different aspects with different results.
reply
And in the meantime, AI would have leapt one more order of magnitude forward?
reply