upvote
> and learned of the general approach they were taking. Only after learning the secret to cracking the problem did they send the first prompt.

Do you have any evidence of this? They don't dispute the timeline, but they never said they knew what Levant/Buckmaster were doing.

reply
It's in OpenAI's first announcement that they had solved the problem.
reply
> Only after learning the secret to cracking the problem did they send the first prompt.

Which quote in the announcement post provides evidence for the above quote?

reply
“ On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model, we launched an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems.”

- https://openai.com/index/navier-stokes-solution/

They do not explicitly admit to knowing about NS specifically, but are extremely explicit that they tried to scoop some potential millennium prize winners.

reply
So then they DIDN'T "learn the secret to cracking the problem". They simply knew that part of the problem was solved. Knowing a problem can be solved and knowing the solution are not the same thing.
reply
The claim that OpenAI somehow used the mathematicians' ideas to leapfrog them seems unsupported at this time and IMHO it was irresponsible to bring it up because credulous people will immediately believe that narrative.

And from my perspective, if some math folks typing in a few questions to OpenAI provides sufficient training data for OpenAI to solve a big problem... that's amazing! A few conversations/prompts out of the billions that OpenAI trains on lead to this result- that means there is an awful lot of low-hanging fruit that could be exploited cheaply.

reply
The (unprovable, yes, without OpenAI being willingly transparent) argument is that openAI constructed a prompt to scoop them using some inside knowledge about the approach, which they allude to in the announcement.

In the transcripts, Brubeck is very cagey and evasive about the prompt, when it was supplied, and its contents.

reply
I'm curious how many other 300 billion output tokens OpenAI has "paid for" that have resulted in no breakthroughs.

Either they had a pretty good idea that investing this type of money in that compute on a model in training would lead to these specific results, or they gambled with other people's money.

I want to hear about the gambles and expenditures they don't brag about. In America's energy economy, there's finite resources to expend.

reply
So because they didn’t admit to it they didn’t do it?
reply
I like how the comment below summarizes it:

> learning the answer might be in model X’s training data made them believe that model X specifically might be able to solve the question, and they were able to very quickly find enough certainty about the former to commit millions of dollars to the latter.

They don’t need to know, because their IP stealing machine knows for them. They just have to buy enough compute, and someone else’s work is theirs.

reply
I said “very quickly find enough certainty” to suggest hypothetical situations like “someone searches the conversation logs, confirms for themselves the solution is present, then shares the confidence gained from this knowledge without explicitly sharing the knowledge itself”. That person could recuse themselves from the project so the project can still legally make claims like “conversation data was not used” in the announcement, while also knowing that they are guaranteed to get there if they just pull the lever enough.

(Naturally, I have far too much respect for OpenAI’s legal team to suggest this is what happened in their project.)

reply
Yeah, and suckers are born every day...
reply
I think you’re overlooking what I’m implying here. It’s not that they knew contamination was possible but they went ahead anyway. To spell it out just a little bit more: learning the answer might be in model X’s training data made them believe that model X specifically might be able to solve the question, and they were able to very quickly find enough certainty about the former to commit millions of dollars to the latter.
reply
> the secret

So such thing existed. In fact, what they learnt was some progress existed, not what the specific progress was.

reply
> They intentionally left Buckmaster and Alpöge out of the citations.

No, they asked if they could do a joint publish.

reply
No, they asked one guy to do a joint publish conditioned on leaving the other collaborator out, with veiled threats. The joint publish part smells awfully like admission of guilt given there’s absolutely no reason to do it if you believe you independently arrived at the result using only public prior work. The leaving out collaborator part is outright academic malpractice. Disclosure: I was an academic once.
reply
To add: with a requirement that he rewrite the proof to credit OpenAI.
reply
> but I can believe it to be accidental

What accident is it when the system is designed to function that way?

reply
Their claim is that training on their solution is "unlikely but possible".

Consider this scenario.

Has a google crawler read my new novel, which I may or may not have posted on my blog, page by page, as I wrote it?

Can you, without knowledge of what I have actually done, claim that the google crawler has not seen the novel?

Without any evidence that I have posted the novel online, it might be tempting to say that the crawler has not seen the novel, but what if I were in an adversarial position against Google on this topic and were challenging them to make that claim. You would wonder if I were hoping Google to overreach by making a definitive claim without taking into account some action that they had no knowledge of. It becomes difficult to use the scientific expression "There is no evidence for this" when there is an accusation of malfeasance because it can be so easily be conflated as "You can't prove we did it". It seems like the best you could say would be 'Unlikely, but possible'

reply
I'm not taking them at their word, sorry. Genuinely, there is no reason to.
reply
> They intentionally threw $15 million in compute at the problem

what? really?

reply
Yes. Maybe much more:

> Such intensive use of AI doesn't come cheap. In a post on X, LisanBench, an LLM benchmark evaluator, estimated that the output tokens alone would cost about $6.5 million at OpenAI's average consumer price. Including the far larger volume of input tokens, the post estimated the total could reach $10 million to $40 million.

https://www.businessinsider.com/openai-math-problem-solved-t...

reply
That's their API pricing. There's no way they actually paid $15M in compute. I'd say much more likely it's in the order of $1M.
reply
Who are you who is so wise in the ways of a private company's internal cost accounting
reply
But think of all the IPO Monopoly money they just generated.
reply
When I worked at Google, we spent $100M in power on protein folding and drug discovery (this was long before AlphaFold). Never underestimate the willingness of smart rich people to invest in speculative science.
reply