upvote
That is an absolutely terrible benchmark. Inference over a bounded search space is not a good measure of what "intelligence" actually is. part of the reason they are using math and not something actually challenging like long distance interstate trucking is because it's so much simpler and easier than what make intelligence intelligent.
reply
Just seems very weird to call getting Fields-medal-level results "inference over a bounded search space" and "not actually challenging".
reply
Claude Fable recently proved the existence of complex structures over S^6 (6-sphere).

If I had to guess, I think LLMs will be inventing highly original new mathematics within the next year. I think it will be approached as an optimisation problem, targeting how quickly LLMs can solve classes of maths problems as a function of the definitions they need to conjure up to do so.

reply