upvote
Copying my comment from that discussion:

Those theoretical physics papers look like the real thing (i.e. not mathematical physics like the OpenAI Yang-Mills result). It looks like AI is finally branching out of math.

https://news.ycombinator.com/item?id=50430239

reply
> It looks like AI is finally branching out of math.

Is it really, though? It's still a heavily mathematical domain, rather than one involving empirical research, where AI still struggles.

reply
In my (limited) experience it does not struggle with empirical research — at least in the procedural aspects thereof — as long as it has access to the right data.

It is very good at understanding research goals, identifying appropriate datasets to work with, doing data wrangling and data science, determining appropriate metrics, running simulations, analyzing results, writing up reports with charts and tables, and proposing follow ups or “high potential next steps” which show a surprising amount of judgment, and if instructed to, executing on those too.

I use Astra and Fable (who delegate to cheaper models for coding) and have them review each others work, and only spot check things when they are done calling each other out. I rarely find issues, and when I do, it’s usually a misunderstanding of intentions.

I’ve only explored a few areas though, including computer vision, vector search, and wireless signal processing, all in applications that, as far as I can tell, are genuinely novel (though of varying levels of novelty and economic value, as the models are fond of pointing out to me!)

I prefer being in the loop and setting the direction — because I do not have unlimited tokens, hah! —- and I regularly identify new directions they had not considered or had prematurely dismissed… so I am a bit more useful than just a financier of tokens! But I can totally see how they could grinding ahead by themselves and brute force useful results, especially if given very concrete goals like a theorem to prove.

I don’t think we’re at Recursive Self Improvement yet, but I don’t doubt for a minute that this is behind the insane pace of development at Anthropic and OpenAI.

reply
In empirical research, getting the right data represents the bulk of the intellectual effort. Designing the right experiment and collecting good data is painstaking, backbreaking work.
reply
I agree, and this may be nitpicking, but would it not be more accurate to say “bulk of the effort” rather than “intellectual effort”?

My perception is that most data collection, once designed (which does take considerable thought, as you point out), is a slog of executing rather mechanical processes over a long time. (Which is what RA’s and lab assistants are for!) For sure, regular analysis is needed to ensure the right data is being collected and no confounding factors are sneaking in, but seems to me the majority of time goes into the relatively rote work of setting up equipment, running experiments and recording the results.

I do think that with access to MCP-enabled equipment, AI would be ideal for this too, given its infinite patience. The problem, as with all other fields, is this would rob novices the opportunity of building the necessary intuition that comes from spending time in the weeds.

reply
I'd argue that "the right data" should really be called "the right information in context."

Context is king.

reply
I agree, but it is the kind of math done the physicists' way (which I found recently asserted that AI cannot do) and not the kind which is formalizable in Lean.

"It's not much, but it's honest work."

reply
From skimming the papers, it’s not clear to me what it is actually doing outside of proving what can be considered mathematical theorems (even if that’s done the physicist way). It doesn’t seem to be making testable predictions (but maybe I missed those).
reply
> and not the kind which is formalizable in Lean.

For people not in the know, it's common - both now and historically - for physicists to use "mathematical techniques" that have not been proven valid.

I took grad level physics courses, and this drove me bonkers. I was accustomed to the comfort mathematics provided me. If something was unclear, I could always dig deeper and find a proof. But a lot of the stuff the physics professors would do - I couldn't, and nor could they. They just responded with "intuition". Some of them had dug deep earlier in their careers and stated that the technique was likely invalid mathematically, "but still useful in getting answers experiments can verify".

I won't even talk about relying on diverging series...

reply
Can you declare theorems in Lean for those techniques. And list them in the paper.
reply
Why would it not be formalizable in Lean? People are formalizing parts of linguistics in Lean.
reply
Most famously: to a physicist, the sum of all natural numbers is -1/12 https://en.wikipedia.org/wiki/1_%2B_2_%2B_3_%2B_4_%2B_%E2%8B...

If you attempt to formalize math involving such assumptions, one expects to encounter a proliferation of contradictions. So I’m curious to see how the contributions of AI authors is handled by the physics community. With pure math, you can ostensibly write a compact statement of a theorem in Lean, and if so, trust the process.

reply
OK, that goes right past me. However, the reference, in the wiki article, to https://en.wikipedia.org/wiki/Monstrous_moonshine seems appropriate.
reply
We used to joke in my physics undergrad that they didn’t need a math department because we were doing all the stuff they were doing but with actual applications. Then the mechanical engineers said the same thing about us!
reply
One of the most efficient ways to torture a pure mathematician is to force him to watch how physicists treat math.
reply
I have strong, complicated feelings about it, and I understand you may not feel the same way, but "I like" is not a good description in my case.
reply
What do you not like?
reply
"I do not like" is also not a good description.
reply
Maybe “thrilled” except in the sense that you’re in a SciFi thriller with a giant alien spaceship hovering overhead in the sky, in much the same way bricks don’t, and you don’t know if they’re friendly or hostile or just doing a survey for their new hyperspatial express route ;-)
reply
I think this is due to the unrelatedness to our universe.

> https://news.ycombinator.com/item?id=50563604

reply
> I like that we're now at a point where "solved two open problems in the physics of black holes" is the kind of thing that shows up in a frontier model launch.

Because they want you to think that you will be able to do the same, too.

Except, you don't have a dedicated cluster for a week with an unrestricted model.

reply
or an IPO launch
reply