upvote
Not in this case, what we needed was a genius mathematician to explore with Claude. Which is exactly what Terry said: "So maybe some combination of humans and AIs in 2050 or something will make further progress."
reply
I remember when, for a brief period of time, the best chess players were human + computer. There were a lot of Ted talks and think pieces reflecting on this reality. This was called centaur chess.

This lasted for 2005–2014. By 2015, the advantage humans added was negative.

I imagine this will be the case here as well, but I highly doubt it'll take that long.

reply
yes, for every specific task. but life is not like chess. life is like playing a million different chess games simulaniously, all of the time. Being a gentle hand that knows when AI is onto something and when it's torturing itself in a loop. He calls it out in the pdf, the aha moments he experiences, that lets him further guide the progress.
reply
Math, specifically, is a bit more like chess though I think.
reply
It's not 2050 yet, though. I feel like the future date he threw out was a big part of his conjecture, that even with human+AI today, it wouldn't be possible, and getting it done that way would take years, possibly decades, of more work on the AI.
reply
And he said maybe humans+AI will _make progress_ in 2050. That is very different from one human + AI solving it 36 days later, lol.
reply
How about we hold off until it's verified? Now, Terence Tao is getting visibly thin these days, I hope he's not torturing himself in an existential crisis. I know I would be. But I would also, in the end, be satisfied as a Lee Sedol figure, the last of the human geniuses, as it were.
reply
As of your comment, it had already been verified by 4+ independent Lean kernels, one of them ("con-leche") itself formally verified. https://leanprover.zulipchat.com/#narrow/channel/113488-gene...
reply
deleted
reply
It looks like >99% of the work is the LLM. He even admits as much.

The genius here is Claude.

reply
Anthropic would have claimed the result for themselves, without Levant, if they could. They didn’t. A strong mathematician was evidently required.

It doesn't make sense to compare the quantity of work. Humans are significantly more efficient than AI, so little work can go a lot further.

I’m sure for every incredible insight which Levant credits to Claude, there’s 100x+ more bad insights that Levant had to filter out.

reply
> I’m sure for every incredible insight which Levant credits to Claude, there’s 100x+ more bad insights that Levant had to filter out.

This is certainly my experience of using Claude every day, in a much less cutting edge domain.

And Claude itself cannot tell the difference between the two. The best it can do is use some sort of external constraint (e.g. verification) to filter them.

reply
Honestly, ever since Opus 5.5 dropped about two months ago, Claude's taste has been great, better than mine, even. Compared to Opus 5, I've barely had to filter anything out by hand.
reply
It is fascinating. Because sometimes my experience is like yours, but other time, like today, I am like “I can’t believe how stupid Opus 5.5 or even Fable 5.2 is”. They will make horrible design decisions, overcomplicating. I will correct and they are like “Of course, your solution is much simpler and better than mine”.

And then they are simply boldly wrong at times. About mathematical facts. That’s why we need formalisation when working with them. They are still stupid and can’t be trusted. But theyvare also smart and can come with amazing insight. It’s a tool, but definitely one which cuts both ways.

reply
I think it's funny that people gauge their personal experiences with AI against something like this.

Do we even know what Alpöge had at his disposal? I think most people underestimate what working with an agent swarm even with the current public technology would be like. We can't assume he didn't have something more advanced either.

I don't feel confident my usage of AI on a personal level reflects any of what has been done here.

reply
Your argument is unfalsifiable speculation. I don't at all that it's self-evident that a strong mathematician was required. Your everyday Joe isn't exactly interested in the Reimann hypothesis so statistically a mathematician would be the one who'd use AI to come up with the solution.
reply
How would a non-mathematician even be able to comprehend the output? Half the words in the pdf are gibberish to me, I could not tell if whatever Claude gave me for a "solve the Reimann hypothesis, make no mistakes" prompt would even make sense, much less be correct.
reply
I don’t know shit about HPC and LLM inference, didn’t stop me from creating an inference engine: https://github.com/computerex/dlgo

A non math person may not be able to comprehend the output, but even professional mathematicians have trouble reading through the huge “alien” AI generated LEAN formalised proofs.

reply
deleted
reply
The real genius is that we have found a generic way of solving problems through brute force by having developed an effective way of aiming an unbelievable amount of computing power at them.
reply
How to brute force 15MLOC to compile is an exercise for the reader.
reply
Reading his post, this is not the impression that I get.
reply
If the genius here is Claude and Claude did 99% of the work then isn't it a staggering coincidence that this was achieved by one of the most qualified and likely humans to have solved this on their own merit and not some random person sat in a bedroom somewhere?
reply
Company selling shovels says shovels did all of the work
reply
Human who could claim credit claims AI did all of the work.
reply
I also use Claude and it never produced this result for me.
reply
Did you also have access to unreleased models by Anthropic?
reply
Doesn't matter; Claude ends up taking cues from whoever it's working with, such that two people doing the same thing get very different outcomes. Claude will always be able to write better software when prompted by a senior engineer; it will always produce better math when prompted by a talented mathemetician.
reply
But also an employee of the shovel company.
reply
Not sure anyone would have been able to write 250k pages of Lean without the shovel
reply
Real men used to be able to fly to moon on 10kb of memory (and 100 billion dollars)
reply
And they weren't able to prove the Riemann hypothesis
reply
But they landed on the moon.
reply
Yet some people are still skeptical about that.

On that topic, I'm surprised the usual anti-ai trolls aren't posting low-effort drive-by "ai slop", "stochastic parrot", "autocomplete on steroids" comments to this very discussion.

Edit: I stand corrected: RunSet, ToucanLoucan, luka598, and others have checked in.

reply
"Maybe 24 years from now we'll make progress" is a little different than "actually it will be solved a month from now with current techniques"
reply
Last I checked it's 2026. Not 20250. And for the record, there is no evidence that suggests that a layman couldn't have made progress with just using AI.
reply
Do we know that that is the case?

We know the architectures of the open weight models as they include architecture diagrams, support in projects like llama.cpp, etc.

Companies like Anthropic and OpenAI don't release information about their models (aside from general things like context window size, whether they support reasoning, etc.). A lot of what we know is speculation. From press releases Anthropic have stated that they are getting Claude to assist with designing new iterations of the models, but no specific details.

The open weight models and other arXiv papers/research are iterating on and experimenting with the base/core transformer architecture:

- RoPE/NoPE/etc. for context window scaling;

- dense vs mixture of experts models;

- Mamba-based State Space Models (falcon models, etc.);

- reasoning, combining transformers with a residual stream, Attention Residuals (Kimi), etc.

- activation functions: ReLU; SwiGLU; etc.

- etc.

reply
I think a lot of model architecture changes don't move the needle ~that much and are basically done to improve runtime performance and scaling. - There's tons of cases of ml researchers, tech companies, and labs that propose new and improved models that turn out to not really be significant quality improvements (if at all) once the community starts experimenting with them.

it does seem like (IMO) the biggest leaps have basically been perf & scaling + RL on high quality data + harnesses.

reply
Yes, but "improving runtime performance and scaling": does directly translate into huge quality improvement too, because then you can you a much larger or deeper model, train it for much longer, and have a larger context.
reply
It's usually people who are not suffiently familiar with the topic, in this case, technology. He has been saying boring things like that about AI in the recent years, and seems to be continuously adapting his positions. I have received negative votes over my criticism of his remarks.
reply
This was literally Dario's contribution to the AI race.

He just did the math and realized you just needed more and more compute to get more intelligence. People keep doubting it but it's showing no sign of being incorrect.

reply
When you say more compute are you referring more to just inference compute or training?

ChatGPT was the turning point of realizing we needed just more compute

reply
We have scaled on both axes and many others, I'd say.

Inference can be increased vertically (a single model thinking for longer), or horizontally (NS was solved by a 10,000-agent swarm).

And pretraining has two inputs that can be scaled independently - training data and model size. (Things like the Chinchilla 20:1 law are meant to establish the optimal way to train with a limited budget - you could just keep making either term bigger and bigger on its own if you had infinite money.)

I think creating and curating high-quality datasets is also part of scaling. Datasets like FineWeb are quite a bit better than the barely-filtered CommonCrawl dumps GPT-3 trained on, but someone had to invest in making them. (To say nothing of whatever exotic bespoke robotics/physics/etc datasets OpenAI and Anthropic are investing in.)

reply
ChatGPT was only the turning point for public awareness (and the massive influx of institutional investment that followed), but researchers like Ilya, Alec Radford, and Dario were already convinced years before ChatGPT that scaling compute was the way forward. ChatGPT just helped unlock the resources (by proving the commercial potential to investors and the subsequent influx of capital) for these guys to pursue that vision at a much larger scale.
reply
I mean, why wouldn't that be the default assumption, if you know anything about computing history? ChatGPT's (v1) contribution wasn't proving commercial potential but demonstrating efficient scaling, due to a confluence of technical developments. If you're talking about v3 then sure.
reply
> ChatGPT was the turning point of realizing we needed just more compute

Ilya Sutskever has been saying more compute = better perf since at least 2012.

  ... the real conclusion now is that if you have a large big data set and you train a very large deep neural network success is guaranteed ...
https://www.youtube.com/watch?v=-uyXE7dY5H0&t=1185

  In the end, the network's size is limited mainly by the amount of memory available on current GPUs and by the amount of training time that we are willing to tolerate ... All of our experiments suggest that our results can be improved simply by waiting for faster GPUs and bigger datasets ...
https://proceedings.neurips.cc/paper_files/paper/2012/file/c...
reply
I was downing some extra bitter coffee while I read that.
reply
Sounds like a valuable lesson.
reply
A spoonfull of syntactic sugar helps the semantics go down.
reply
deleted
reply
current AI might not be optimal, but still a powerful unoptimal could create optimal AI
reply
People are so goddamn impatient. They demand miracles by the end of the week, otherwise they call a flop.

As if drawing a line through few points on a chart is a mystic art.

reply