And no new releases from Chinese labs likely means their models that they could compare against those models are still baking largely because they lack compute so severely they need to train models for far longer, or figure out training on Chinese hardware.
Either way it's clear open models a lagging by ~1-2 months compare to frontier, depending on what frontier means to a person. Honestly most new models are just not that much better than at edge cases, they still fail a lot for the kind of stuff I do, but those edge cases matter a lot for un-interrupted long horizon work.
I personally am starting to see a wall here in llm development but that might just be me, given that people are being forced to scale models up considerably for frontier performance.
It's scary thinking that modern models will likely be at best 2-3% better in benchmarks or only useful in 1-2 unique edge cases compared to base models for 100x cost increase in compute required, it's getting absurd.
I can't imagine how Fable models can retain any value once you have to go 100x larger, while 27-32B models start to catch up to the current frontier for coding or other narrow/focused tasks in a year or so.
Lots of synthetic data, distilled data etc.
The problem is if we can pack a Fable in Qwen 4 27B who will pay subscription prices?
I fully expect that we will see increased performance from smaller architectures (to a point) simply because they are refined using larger ones.
> The problem is if we can pack a Fable in Qwen 4 27B who will pay subscription prices?
This is available to everyone developing these systems. Prices per relative unit of intelligence will continue to decrease. And market economics will continue to drive the cost down.
Ala intelligence is a function of compute, there was no guarantee that you need to 10x compute for 10% better performance or something like that.
If you ask the labs they will say the scaling is fairly linear or atleast sub-quadratic, they definitely won't say it's exponential.
No one was thinking we need to 10x compute from here again for another 10-20% increase in performance.
It's only been since Opus 4.6 that the compute growth vs improvements are starting to feel this way.
For instance GPT-3 vs GPT-4 was practically a 10-100x jump for a 10-100x increase in compute, we went from broken sentences to fully formed essays writing, speech and so on.
GPT-4 vs GPT-5 was just as massive we went from no agentic capabilities to fully agentic, 10-100x for a 10-100x increase in compute usage.
GPT-5 vs GPT-6 was atleast a 10x (upto 100x) increase in compute, but the difference is not that clear sure people say that one is significantly better than the other, but GPT-5.5 was good enough compared to where we are today.
Even on benchmarks compared to GPT-5.2 Codex we have almost 2x'ed the performance on say artificial analysis (53 vs 29). This is absurd. 10-100x increase in compute for 2-5x performance improvement is a massive gap.
[1]: https://bitterlesson.ai/ [2]: https://artificialanalysis.ai/?models=gpt-6-astra%2Cgpt-6-lu...