Last weekend I spun up a small agent swarm and pointed it at a field of math I have some affinity towards. Within four hours I had settled three conjectures, one of which is rather famous (for the field, not in general). It cost me about four hundred dollars.
I am at a loss about what to do with these results. On one hand I feel like the mathematicians working on these should know about them, but on the other I feel a bit like a barbarian who suddenly finds themselves sacking Rome.
It's also possible that the result is already known and you just weren't aware of it. It's easy for someone outside of a field, or even one steeped in it, to not be aware of certain solutions.
I actually partially disagree with this. What happened to all the excitement about Intelligence Augmentation (IA)? Now it's AI instead of IA. I think there's so much untapped potential for augmenting our intellect with the likes of https://dynamicland.org and https://folk.computer, as well as the work that's been going on in college math education, things like Lean, etc. I think the only reason human math capabilities haven't expanded that much is a failure of our imagination, not our potential.
I would recommend publishing them to Palomar (https://palomar-registry.org/) - I have no affiliation, this is an online registry of Lean-verified proofs created by Terrence Tao.
I have submitted a proof there that's also minorly important in an extremely niche field.
Anyway, I feel like it's a good place to dump AI slop lean proofs because the main point of the registry is that it verifies that: 1) your Lean challenge statement is the same as what you informally state you're trying to prove; 2) your Lean proof actually compiles.
This could be useful to future AI slop researchers who want to know if a given result has already been formalized, and they may be able to mine some lemmas from your work. Also, it's good to know for the field in general what has been proven.
I'm fairly certain you can set your publishing name to be whatever you want, so you could set it to be just the word "Anonymous", or the name of the model you used.
You asked for a painting. A robot made the painting. You looked at it and said, "well, I guess it's good. Should I put it online or something? Dunno. Hey Fred, what do you think of this?"
Meanwhile, your next door neighbor spends their entire life developing their understanding of life through art. They "understand" (maybe not in a way they can articulate) art. You go next door, you look at their painting and say, "well I guess it's good." But you also understand that your neighbor is just like you, and maybe you are a painter in another way.
I find it strange that, people can't see that, we don't need to solve hunger and poverty and work balance, and etc, by a round-about make-super-intelligent-AI. We could just solve it. It's pretty obvious how to, as well.
We can all be painters, if we put restrictions on the psychopaths.
They're valid.
The biggest problem is, IMO, drivebys uninterested in actual results, just getting a check mark, and the equivalent of dropping a 200k line PR on people and expecting them to be interested and do the work for you. These are things many on HN are familiar with and know how to do better :)
I can understand why the community is pissed. So now, lean proofs can be churned out at scale, and the community is left to decipher all of that slop into human understanding. There are bad actors with misaligned incentives coming in with drive-by proofs upending what the community holds dear which is to practice and propagate the art. I applaud them for this declaration.
To re-align incentives the following could happen. AI slop lean proofs are dumped unceremoniously into a lean dumpster, and what gets rewarded are results that could digested into human understanding - via the already followed human review process. Prizes are not given to lean proofs since anyone with sufficient compute can churn them out.
If you are trying to understand better the field, then do a good write up of the proofs so that people can learn from it.
If you want to earn the respect of people because you found interesting proofs. Then do a good write up of the proofs so thst people can learn from it.
If you want to plant flags and pollute peoples minds. Then please publish it anonimously, no one wants to correct LLM slop for you.
Probably we should build a repository of AI slop proofs that are only allowed to be publish anonimously. That way people may be more inclined to work on it because they would feel like they are cleaning your house for free.
I like my current life and don't want to get dragged into the current fracas surrounding the use of AI in math.
The problem is with people that may do it without contributing to the community.
The field I've been investigating is not large. Even if I were to take the time and care to beat the interesting results into something meaningful, I'm afraid the pace at which I'm able to produce these results would not be well received.
Like the mathematicians working on famous problems in private until they could claim full credit for something interesting wasn't also a marketing exercise for their own careers. The commercial value (or lack thereof) of a proof doesn't depend on whether it was done by a human or a machine.
OpenAI just burned millions of dollars over a weekend after hearing that someone else was close to solving the problems. Their interest was in their AI system more than the actual math problems.
Don’t you see how that’s different?
If clout was the goal I don’t think becoming a lifelong mathematics academic would be the first step
I guess that type of “clout” feels different to me.
Wanting to be validated by peers for your talents in a niche field vs. using millions to try to solve a math problem that you don’t really care about with AI to market the gigantic company you work for.
I can see how what is happening to mathematicians is similar to what is happening to coding.
I’m not sure what your broader point is? Mathematicians shouldn’t be upset? Coders should? Something else?
From your first comment it seemed like you were disagreeing with me but I’m not sure how.
That’s why I asked for clarification.
Edit: BTW I’m a web developer and designer, Not a mathematician
This is the core misunderstanding that the open letter is attempting to correct.
Developing a better understanding of the Navier-Stokes equations could have a number of implications for useful technology. They're fundamental to fluid dynamics, and turbulence in particular is something that many people feel we could work with more effectively if we better understood how and why it's generated. The Navier-Stokes smoothness problem is an interesting and long-standing benchmark for this understanding; we don't know why it should be so hard to answer, so we hoped that the process of developing a proof to the problem would produce more understanding. (We may still be able to extract this understanding after the fact, if OpenAI's proof is fully human-comprehensible.)
Simply knowing that there exists a finite-time blowup is not practically useful. We know that fluids in the real world don't produce random singularities, so the result can't really have much physical meaning. What it illustrates is that the Navier-Stokes equations fail to model physical fluids in some yet to be characterized way.
How much do you think other AI companies would offer to get access to the transcripts of the generation that led to the proof? No doubt OpenAI will include it in their training data somehow and use it to build the next generation.
There is already economic value.
One of the points the parent makes, along with the TFA, is that academia -- or more specifically, the "mathematical community"-- is a setting primarily for creating and ingesting mathematical knowledge, and disseminating it to the next generation and to other fields. Humans absorb this material slowly, through lots of discussion and collaboration -- it is necessarily a slow process. Facilitating this is one of the important functions of academia. Your usage of academic as a slur here is a bit silly for this exact reason.
I don't claim it is perfect, and we can argue about pedagogy in elementary courses till the cows come home. That's not really material. But this is one of the only settings in which such knowledge is broadly valued for its own sake, and in which there is a semblance of incentive to help others "know" this stuff as well, be they future generations of mathematicians, science and math educators and communicators, practitioners in other fields, or genuinely curious amateurs.
The point is that these proofs are largely useless without the insights. The value of a proof is largely in the travel, not so much in the destination.
Okay, to elaborate, substantively, their point is that the people using these AI models are not doing it for the love of the game, but for marketing. And instead of them - and nobody - spending millions of dollars to solve the problem, successfully, they want every problem of their academic industry to persist because even though they never solve the problem, they synthesize and solve lots of other problems nobody asked for. And get to boost their egos?
Yeah, stop that. Actual alignment is on the humans themselves, if they want to remain relevant as academics and mathematicians, they need to learn how to replicate the proofs and the steps that alluded humans for decades and don't worry about the narcissistic elements that slow their industry down.
The NS counterxample is actually, by any market measure, a "problem nobody asked for" in the sense that its existence doesn't have any commercial relevance (beyond juicing OpenAI's IPO). So the only long-term value solving it could have is by virtue of whatever reusable theory/insights were generated along the way to the counterexample itself. The letter is absolutely right on that point.
It's not actually clear that those insights will come faster from reverse engineering this LLM proof vs. humans building theory to solve the problem themselves. So what you're saying may or may not even be an efficient way of operating. Also, it implicitly depends on mathematicians to do the hard work of creating problems and then deciphering LLM hieroglyphics for essentially free while the only immediately profitable component gets outsourced to a frontier lab. In what world is that model going to work?
Reading between the lines, it seems like maybe you have a personal grudge for some reason and simply think the technology will advance enough to where we won't need academics at all. But you should say that in the first place.
My stance is that solving the problem is aligned with humankind
the rest is just hypothesizing a way that academics fit in this world at all
I actually found this to be the case with some basic linear algebra notes I was recently doing in Lean (without using mathlib). The model could generate working proofs, but they obscure the basic ideas (actually I wonder somewhat if this is because the Lean code that's out there to train on doesn't make a huge effort to read like textbook proofs, which was my motivation in the first place). I give it a skeleton of a couple lines of `calc`, letting it fill in the reasoning for each line, and it does much better. Then ask it about making some macros to simplify "trivial" or "obvious" things, and it does even better. etc.
I suspect there's a good workflow where a big SOTA model makes an impenetrable proof (or code) and then a human works with a FIM model to simplify it (with the larger gnarly proof right there in context for FIM), but unfortunately everyone seems to only care about agents right now.
Presumably you're a human. Are you going to do that?
To me this analogy points in the complete opposite direction. Imagine somebody takes a half-completed project design you're trying to figure out, vibecodes a rough prototype of it, emails your manager to announce that the project just launched in alpha, and then dumps it back on your lap for approvals and testing and productionization. Would you say that they've added value to this process? Or did they just strip away all the hard parts of the problem so they could claim credit for the easy part?
If that person then runs around telling people that they're the real author of your project, because they generated the original POC, would you consider that an accurate assessment?
But mathematicians define their field. They're smart people. They're capable of recognizing when someone just did a vibecoded throwaway PoC and when someone has a well structured proof. Actually even before LLMs they'd publish new, clearer or more elegant proofs of old results. They can say that inscrutable proofs are exactly as valuable as they are, and that the first explanation people can actually understand carries its own prestige.
This letter includes someone like Terrance Tao who publicly expressed a lot of optimism about AI for solving novel math like with the Erdos problems. It's not sour grapes but the first steps to define those new expectations for the future to reduce the perverse incentives.
And yet, predictably, people are accusing him of "gatekeeping" and ignoring the arguments he has made here and elsewhere about the benefits vs. harm in different ways of using AI.
I'm also not sure I understand what you're objecting to if we agree that mathematicians define their field. The source link is a declaration from 25 Fields Medallists with precisely that goal. They believe/define/declare that the type of AI-generated proofs we've seen are vibecoded throwaway PoCs; they feel that a well-structured proof must include factors such as "a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others", and the success criterion is not a true/false conclusion but rather "development and integration into the mathematical canon".
Is it reasonable for any field to make such demands? If this were doctors objecting to AI becoming good at medical practice would you have the same concerns?
While any idea of OpenAI spying on people to pursue their goals is disgusting, the rest of this is par for the course, as Kasparov experienced with IBM in the 90s. Humans still play chess after all.
I doubt OpenAI will take such a combative stance and accuse these mathematicians of "demanding" things, as you do. As I said, the purpose of this is marketing and the statement simultaneously undermines the value of that marketing (showing these projects as irresponsible) and gives these companies an even better piece of marketing in its place: "our AI got so good at maths the mathematicians begged us to stop". It's entirely possible they will stop pouring millions into these projects.
The fact is these fields are supported by society because of the benefits to everyone else. Once the same results can be achieved in a cheaper and faster way that is what will be done. We should mourn this in the same way we do buggy whip manufacturers. Again people still ride horses.
Would you expect Fields medalists to cure cancer if you moved them from the math department to a medical research lab? This is precisely the fallacy that the frontier labs are counting on to inflate their valuation as their IPO approaches. They want to use headline-grabbing problems in pure maths to make their models look "smart" in the public eye. But what does "smartness" in mathematics really mean in terms of economic value? It is not at all obvious whether success in abstract mathematics should translate to successes and, more importantly, profitability, in more grounded endeavors.
Look at OpenAI's job postings (https://openai.com/careers/search/). Those roles involve far more pedestrian yet profitable duties than research mathematics. So why isn't OpenAI automating them with their vaunted models? Success in one field, no matter how "difficult", does not predict results in another field.
Maybe it's worth double checking that you know how these fields benefit everyone else? Proving the blowup of the Navier Stokes equations in 3D isn't going to make your gas cheaper or make harvesting food easier or make drones easier to protect against. Maybe consider the deeper effects at work?
If you can make breakthroughs on such areas as fluid dynamics, control theory or information theory with AI then that absolutely is a big deal with real technological implications.
It's this sort of thing that motivates people to burn down the institution you might be trying to defend.
lol, yes, this sort of thing is what many people who voted for Trump were saying, and things are going great for them.
> They prove a millenium result, but it doesn't count because they are bad people.
OpenAI has only themselves to blame for this, and they know it. They could have handled this so much better. I'd bet there's more meeting time right now going into how to unveil future math results than on meeting about the actual math research.