upvote
Can confirm that current GPT (with no guidance) took 5 minutes to find what it claims are errors with counterexamples. (I have nowhere near the level of state-of-the-art mathematical knowledge and experience needed to validate them. I'm posting this solely as additional confirmation of "there may be issues with this proof that an actual mathematician together with LLM assistance would be able to properly evaluate".)
reply
I wouldn't be surprised if these models were aligned to extreme skepticism (to the point of hallucination) when e.g. a user proposes a solution to a Millennium Prize Problem.
reply
1) There's a meaningful difference between "I'm not buying it based on my priors" and "I'm claiming to have a counterexample". I'm looking forward to seeing analysis by other mathematicians.

2) Any model with properly updated priors will have less skepticism of a Millennium Prize Problem being solved now.

reply
Did you try the same with the other LLM assisted proofs? I would be curious what your GPT model says about them. I fed Alpöge and Anthropic statement and details to Gemini 4 hydrogen, and it said that there are "major problems and red flags in both the text and the current verification status", and that it is impossible to verify the 250k pages of slop, and there are there are "Extraordinary, unsubstantiated mathematical leaps in the sketch". So I would take these surface level reviews by LLMs with a grain of salt.

Anyhow, I hope that they are both right!

reply
And that is the problem.

AI claims but too few people can check.

Won‘t get better from now on

reply
I mean I guess there is incentive for some human curiosity and trying to learn more than ever, small bit but still to get a standard for verification.
reply
What value is there in just dumping something into an LLM and saying it produced errors?
reply
Very high value. The LLMs are excellent at auditing math proofs.
reply
It's being hurriedly announced as fully human work. I wonder if the author had carefully avoided ever running the full argument through frontier LLMs (OpenAI/Anthropic) for fear of it being stolen or incorporated into AI training.

This may be the first time this proof is being proof-read by frontier models

(this is pure speculation, but provides an answer to your question)

reply
I also suspect the author actively tried to avoid using AI at all.

Another possible reason: AI will correct your mistakes by using ideas of others. A proof will unlikely remain human-only, even if you just use AI to check for your errors.

reply
Have you tried doing this recently? For my work on software at least, the latest LLMs have a very low false positive rate for bugs. They almost never hallucinate issues. My understanding is the same applies to math.
reply
I'm not arguing that LLMs aren't useful tools for mathematicians, I'm questioning the quality of discourse that amounts to "I pasted this into the LLM and it said there are errors".
reply
There's value in knowing whether a claim is true or false. Plugging things into an LLM doesn't necessarily give 100% certainty or tell us why something is true or false in a human-comprehensible way, but it can move our credences and suggest directions for investigation.
reply
If you're not even gonna share what the supposed errors were, the value is pretty much infinitesimal.

Surely, GPT said something more elaborate than "nuh uh".

reply
Think of it as a (fuzzy) compiler.

Now think about complaining that someone "merely pasted my code into a compiler and said that there are errors in my code."

That's nuts, right? If a compiler -- a kind of proof engine -- finds errors, there are errors. Fix them! Don't complain about the usage of a compiler, the errors were real, they're there, they need to be fixed.

The fuzziness part is both useful and a liability, but the error rate is dropping.

Some numbers for you:

A year ago, I ran a huge legacy codebase through Gemini Pro 2.5 and it spat out several thousand bugs, but manual inspection showed that 80% were false positives and only 20% were worth fixing.

I did a similar exercise with GPT 5.6 more recently, and the hit rate was 80% valid and 20% false positive. The human developer got into an argument saying that they thought all 100% were false positives. He lost. The AI could easily produce "repro" snippets that triggered every issue (mostly security vulnerabilities, but some correctness ones too).

These days, if you're publishing anything with verifiable claims in it, and not running it through an AI first, it's the direct equivalent of uploading open source code to GitHub without bothering to check that it compiles first.

It's sloppy.

reply
classic HN isolated demand for rigor. i found it interesting
reply
I would guess that LLMs are much worse at math without having a theorem prover to validate approaches with. Since this draft is not formalized, an LLM is operating purely on the text which is more tenuous.
reply
Pre-chatgpt 5.5 it hallucinated quiet a bit on hard math. Starting from 5.6 it is incredibly good.
reply
I’m doing it daily and it finds a bunch of issues which are not issues in practice.
reply
I suspect LLMs are much better trained and more reliable at bug identification in react apps than draft-state proofs on the bleeding edge of math, on problems that are specifically highlighted as extremely difficult to solve without error in their training.
reply
> the latest LLMs have a very low false positive rate for bugs.

Man people live in different worlds, astra for me finds 25/6 bugs all the time.

reply
Cool story.

I have the opposite experience in my line of work.

reply
What's your line of work?
reply
Seriously. At least be constructive. Share what the errors were. Make an effort to understand them, anything. That tweet just amounts to low effort FUD.
reply
[flagged]
reply
ah yes, the author who has been working on this problem for 12 years is lazy because they didn’t feed their work into the slop machine so it could train off it and steal their work.

this place honestly sucks

reply
It has to be new people that have come here, this place was NEVER this terrible before. Now it seems like the world has conspired to direct the worlds greatest idiots to descend on this poor site.
reply
deleted
reply
Well he clearly lives in the old world. It's a bit like someone not checking a search engine before cleaning some basic fast wrongfully that can be verified in 10 seconds.
reply
not everyone is a slave to the machine, man
reply
And see where that gets you, second place and 11 years down the drain with a half-working mess, congratulations hope the winning of not being a "slave to the machine" is feeling great.
reply
> second place

If the result holds then this is the paper that will get the medal and what history will remember. First place enough in my book.

reply
> but either way, alpoge/anthropic certainly has the claim to the first proof

Is that a thing? Don't people care more about the proof's elegance and insight than its timing?

reply