upvote
I agree with you and I think, personally, that the heavy post-training like here makes the model more capable in coding but less capable for certain unbenchmarkable tasks, like prose or interpreting prompts correctly. I've been noticing that small models have been gradually being pushed towards programming and are slowly becoming _worse_ at some non-programming tasks, not better; e.g. new Deepseek models appear to be much worse at interpreting metaphors and overall feel more autistic than previous ones.
reply
That mirrors my experience exactly. I keep a couple of older models (and much larger ones as well) for when I want to do stuff that is unrelated to computer programming because the models that are really good at that tend to fare less well at other stuff. I suspect that there is a boundary where maximizing benchmarks for coding at the expense of general knowledge is going to give you a model that is really fast and good but 'narrow' and that this may also lead to loss of performance when applied to coding tasks that are less well represented in the training set (such as, for instance, DSP code). This is a variation on 'the bitter lesson', short term you can make gains like this but long term leveraging compute and data will give you something better. I'm really happy I didn't waste half a life on engineered AI because I would be really quite miserable if I had.

http://www.incompleteideas.net/IncIdeas/BitterLesson.html

reply
Why do you find it hard to believe that you can do some RL and improve across the board (albeit less strongly on other semi-related tasks than your objective)? That's the idea behind generalization: it's not perfect but you get a little bit of cross-task generalization, so if you just keep spamming more RL environments you can hill-climb to AGI. That's what the labs are doing!
reply
You can hill climb but you will (or at least, I think you will!) lose generality.

And no, that won't get you to AGI, that is a completely different goal.

reply
If you carefully curate the data, plus the original model isn't saturated for RL post-training, it's possible to improve targeted benchmarks without regression on other metrics. This experiment is done with terminal bench 4.1 as a target, and the regression on other benchmarks has been carefully controlled through data + infra design.
reply
Ok, so the obvious question then is: this seems (I'm not saying it is, it just looks like a small thing compared to training up a model from scratch) a relatively trivial undertaking for the authors of the original model, if there was this much left in the tank why didn't they do something similar? Or is there some step here that only this team can perform? Or is it simply a matter of economics?
reply