upvote
While intuitively it would make sense that smaller models specifically trained on math perform better than large generalist models, this has empirically proven to be untrue.

Deepmind was proving math theorems using a model specifically trained on math, and that was very quickly surpassed by generalist LLMs. It's presently not well understood why this is the case.

reply
I'm surprised that this is surprising for the industry. All knowledge proceeds from the same wisdom. The more fields you master at a very advanced level (eg. STEM, philosophy, marketing, etc...). The more you notice that the same basic principles drive seemingly unrelated fields.

I am not simply talking about applying a mathematical technique commonly used in one field to another. At a deeper level, you always get back to playing the same game in a different form. For example, sales engineering has a lot in common with cybersecurity.

And this is a good example, as it's extremely rare to find someone with advanced expertise in both fields, since they typically attract two radically different kind of people.

For this reason, where the output expected is not mechanical, i'm not surprised at all that generalist models are doing better than specialized ones for open ended questions and frontier work.

reply
This is definitely a conjecture; it gets to how much SGD will generalize out of the training distribution.

If pretraining alone was sufficient then nobody would do RL. And, heavy RLVR on coding/math is why these models are savant-level in those domains, yet not superhuman at all fuzzy reasoning and practical wisdom tasks that are only represented in the pretraining corpus, and don’t have an RL target.

reply
Correct me if I'm wrong but I thought there were still a number of cases where models trained for specific tasks perform much better than frontier generalized LLMs. E.g. Stockfish and KataGo perform much better at chess and go than the best frontier models (obviously I don't know what the internal models are capable of in these realms).
reply
general training remove noise
reply
I wonder if it's also that it gives them a better sense of analogies. In my experience small LLMs do really poorly with analogies, but the large frontier models seem to get them much, much better.

In math at least I often use analogies to help solve problems, like "ah maybe if I think of this as a vector I might use linear algebra here" and such.

reply
Sounds like a bitter lesson.
reply
Yeah, my model was that pretraining, RL, and test-time all had their own scaling returns, so just looking at pretraining loss perhaps oversimplifies?

I could believe that your RL/test-time scaling curves are strongly constrained by the pretraining loss and so in some sense that makes the max/optimal performance “latent”.

But its not clear that pretraining loss alone gives enough information on capabilities, particularly as we get closer to dangerous thresholds in some dimensions. (Eg CBRN capabilities, to take a mundane harm that’s currently impacting us directly.)

I’d like to see the scaling curves here; intuitively I’d guess you get more return from RL/test time, the smarter the base pretrained checkpoint is; therefore the error bars on capability/perplexity should be getting wider on this chart.

reply
Yes training on math would give you spiky abilities on math. But in this case the dataset they measure perplexity is ood from training so lower perplexity on it indicates a general increase in intelligence across domains.

But there’s definitely spikiness and it’s very possible they’ve trained more on math than other domains.

reply
there's a whole field of science that concerns itself with emergence as a phenomenon, where complex systems suddenly exhibit new dynamics once they cross a certain threshold. stuart kauffman's "the origins of order" is a really nice read if you want to get into the early stages of that kind of science. a lot of work done in that area by biologists trying to explain the origin of life/self-organizing matter. a more recent term from AI research that describes a specific critical change in a network past a certain threshold is "grokking".
reply
why would you be skeptical given that the claims seem to hold up in real world ? Assuming they measure stuff correctly
reply
Because correlation doesn't prove anything? The "real world" might be telling you something but maybe not what you think it is.
reply
Me too. Perplexity of what? If we want models to be able to reason, call tools, or solve new problems, none of those necessarily have existing data on what the next token should be. A lot of advances in frontier models come from training with RL, which is fundamentally different from next-token prediction and doesn't have a notion of perplexity.
reply
Pre-training these days includes datasets with full session rollouts with tool-calls, user/assistant turns, ...
reply
Sure, and learning to predict those tokens is probably a good way to start training a model. But the hope is also to eventually make a model that's _better_ than what's in that data, right?
reply