upvote
This comes up frequently in threads but it's actually a much less impactful issue than you might think. There are actually several entirely disjoint implementations of Lean, each with different bugs; it is incredibly unlikely an agent would be able to exercise the same bug in every single implementation.
reply
Does a malicious proof need to exercise the same bug in all the engines necessarily? Why couldn't it exercise a different bug in each engine?
reply
You trivially could through the principle of explosion.

That said at least one version is itself formally verified, at the machine code level if I understood correctly. It might still make mistakes but you're starting to get to a point that those mistakes become quite unlikely.

It's still a pretty tricky endeavor but we should know soonish, with some confidence if it is correct or not.

reply
"There might be some mistakes"

"No, it has been proven"

"The prover may have mistakes"

"No, the have been verified"

"The verifier may have issues"

"No..."

Ad infinitum

reply
But every round the minimum complexity of any potential exploit is dramatically raised, until the possibility of foul play can be reasonably ignored.
reply
Agrippa's trilemma never fails.
reply
Is that just an informal version of Gödel’s first incompleteness theorem?
reply
I'd argue Gödel's incompleteness theorem is more of a formale example of the same.

Though it is technically different. Mathematicians were happy to pick a set of axioms, they were less happy to learn any theory capable of stating facts about basic arithmetic would necessarily be unable to prove all things it could state.

reply
The successor function might have a bug.
reply
I’ve heard it argued that equality has a bug. In real life it’s a lossy comparision as almost nothing is truly equal (except maybe atomic and subatomic particles?)
reply
You could maybe address this through holding out the other implementations and using them only to verify?
reply
I doubt that any of them are closed-source, so the clankers probably already know all their bugs. Who says they all don't share a similar bug that is necessary for the proof to work? No one is going to review 1.3GB of anything by hand, even if they could understand it all in a single lifetime.
reply
Are there any that don’t rely so heavily on type theory?
reply
you don't want to review the proof, you want to review the verifier.
reply
> so the clankers probably already know all their bugs

I agree. So if the AIs knows some bugs, they will be using them to prove all kind of fake stuff, not only interesting stuff. https://xkcd.com/704/

reply
Some of the reimplementations are actually verified, so guaranteed to be free from the sort of bugs found in the main kernel recently.
reply
All implementations built upon the same spec no?
reply
as per https://xenaproject.wordpress.com/2026/11/12/and-the-proof-i... before ever thinking about kernel bugs, you should think of the very basic "is the lean code even stating the problem, not even to say correctly"
reply
Yeah maybe even more important point
reply
Apparently they use the Mathlib definition of Riemann, which has been reviewed by actual human mathematicians.
reply
And I understand that definition is 16 lines long.
reply
It would have to also be really sneaky about it. A proof that's trying to invoke some bug will look very different from a proof that's the real thing.
reply
I found a bug in Lean a couple of weeks ago with an LLM, in fact, but it was in its 'macro' system that expands code into statements, not the actual proof system. It wouldn't have proven anything false, but it could have produced statements that aren't what you intended.
reply
Yes, this is starting to look like the infinite monkey theorem.
reply
...if it's tasked to find proofs of False assuming that there are still bugs left.
reply