It doesn't make sense to compare the quantity of work. Humans are significantly more efficient than AI, so little work can go a lot further.
I’m sure for every incredible insight which Levant credits to Claude, there’s 100x+ more bad insights that Levant had to filter out.
This is certainly my experience of using Claude every day, in a much less cutting edge domain.
And Claude itself cannot tell the difference between the two. The best it can do is use some sort of external constraint (e.g. verification) to filter them.
And then they are simply boldly wrong at times. About mathematical facts. That’s why we need formalisation when working with them. They are still stupid and can’t be trusted. But theyvare also smart and can come with amazing insight. It’s a tool, but definitely one which cuts both ways.
Do we even know what Alpöge had at his disposal? I think most people underestimate what working with an agent swarm even with the current public technology would be like. We can't assume he didn't have something more advanced either.
I don't feel confident my usage of AI on a personal level reflects any of what has been done here.
A non math person may not be able to comprehend the output, but even professional mathematicians have trouble reading through the huge “alien” AI generated LEAN formalised proofs.
On that topic, I'm surprised the usual anti-ai trolls aren't posting low-effort drive-by "ai slop", "stochastic parrot", "autocomplete on steroids" comments to this very discussion.
Edit: I stand corrected: RunSet, ToucanLoucan, luka598, and others have checked in.