upvote
LLMs are already multimodal, though? Plenty support native image inputs for instance.

But also: human physicists mostly act on symbols, too. They take measurements using sensors, which produce a bunch of numbers, that they then tease theories out of. This seems in LLMs' wheelhouse.

reply
This is what people seem to be missing when they extrapolate all LLM mathematical brute force discoveries to any X or Y scientific problem. Most real-world stuff is not as cut and dry as formal logic and math. Computer science is applied mathematics and almost all logic-based, it follows that LLMs would be good at this. Lot harder to model proteins or materials science this way.

That said the stock price juicing counts on you seeing these discoveries and then attempting to extrapolate bespoke frontier models' success in math (which is dubious in terms of methodology and tailored training anyways) and immediately jumping to the conclusion that AGI and huge scientific advances are right around the corner and can have us all living in fully automated luxury Yudkowsky MIRI effective altruist utopia. It's a farce.

And yes, someone is going to comment on this by saying "moving the goalposts, in 2022 models couldn't even do this." Yes, it's incredible how much throwing money at certain logically achievable outcomes helps achieve those outcomes. It is not clear how it helps achieve random wish casting by odd cult-like CEOs, however.

reply