It’s not quite as simple as that. Several studies have shown the opposite: models trained on more diverse knowledge tend to cross-pollinate across domains. So a more generalized model can actually perform better than a specialized one.
That’s why you’re not seeing tons of tiny models (one for Python, one for Pascal, one for Rust, etc).
But it doesn't match my experience. Qwen3.8 27b is clearly smarter at coding than MANY bigger models. gpt-oss-120b for example, is almost 4x the size, and performs way worse at coding tasks.
It's clear to me that you can build small models that work well at specific tasks.
Python vs Rust is probably too fine grained a way to build a model. Coding in general seems like a better target.
There will always be a place for large generalist models, no doubt. But I think that place is much smaller than the big ai companies are counting on.
Shows qwen3.8-27b along side seven larger models of ~similar vintage. Only one scores above 27b.
Many of those are closed models so idk their exact parameter count / active param count, but it hardly matters - i’m sure all of them are far above 100b params
My point is not that bigger is pointless. It’s just clearly not the only road to take to make a model better, which is obvious just from seeing how models of the same size have gotten better over the past few years
First off, I'd include Qwen flash-next and GLM 5.3 to show some of the other strong open weight models, and they predictably dominate it, but they're much larger. But, it shows up right next to DSv4 Flash 0731 on the overall index, and that's much larger. It's a great model! But then scroll down and hit Time Per Task, and you'll see that DSv4 Flash takes 3.6 seconds per task to Qwen's 21.1. That's what I meant when I said this:
>speed due to excessive thinking maybe to make up for the smaller amount of world knowledge baked in (qwen 27b's main issue iirc), etc - they're tuned for different things.
It can make up for its shortcomings by iterating a lot longer, and using way more thinking tokens. And that's a great trade if you don't have the vram to run the bigger models, but speed is pretty important for getting things done... And that's why DSv4Flash is great, too, despite being much larger, and scoring similarly on the intelligence index.
It practically became a joke about how a huge amount of the training data for GPT-4 was bottom of the barrel reddit vomit and obvious bot spam. Leading to many bizarre edge cases.
I make heavy use of smaller local models on a daily basis (Qwen3-VL for auto-captioning images, Gemma3:27b for some translation work, etc.). Gemma3:27b is a good example of a very capable general purpose multimodal model and has handled almost everything I've thrown at it from sentiment analysis to documentation writing.
I suppose I was drawing a distinction between specialized and general intelligence versus small and large. I don’t think those are necessarily mutually exclusive.
And Qwen3.8-27b is still better at coding than opus 4.1.
Yes, if you list off models 27b is better than it’s all older models. But that’s my point - newer models are better than older models at the same AND much smaller size. That’s because model size matters less than they say. Training data and model architecture matter more.
Yeah, the cross domain transfer learning from RL is overstated by a lot.
In 2026, the default outlook should be suspicion for any big private organisations with profit motive.
You don’t need to think about climate change studies. Instead you can read the allegedly tainted studies we’re actually talking about and profess to all of us what is wrong with them. You can’t point to exactly where they’ve fudged them.
And even if compute demand were perfectly elastic it’s only a good thing insofar as it drives demand for new Nvidia hardware. If tokens can be served from Apple hardware or Google hardware or Huawei hardware that doesn’t help Nvidia.
I mean… some homes definitely do. You must have seen those houses that are all lit up front the outside by lawn mounted spotlights.
If that dies because a lot of people’s needs turn out to be met by a system at home they can run a 30b-150b model on, a lot more of that money goes to apple or intel or amd.
1. Yes, smaller models will become more popular, especially as the tokenmaxxing trend dies down and people start stretching their budgets farther. That is a downward pressure on demand.
But along the same dimension, consider that currently only about 40 - 60% of the world uses AI for only about 5 - 15% of their work hours. That means there is still 2x growth from users and 7x - 20x growth from the rest of the work hours left to capture! That is 14x - 40x more demand. Then consider that agentic tasks require multiples more tokens, and that is the kind of usage that is most likely to be deployed, and also the kind of usage that is the least used right now. That's another huge multiple to be tacked on.
And the entire AI industry has been lamenting the extreme compute crunch they're facing (and also why Claude has 9's comparable to GitHub; whereas OpenAI has been chugging along because Altman was OK being called a "podcasting bro" while desperately scrounging for compute years in advance.)
Nvidia's meteoric rise is entirely due to this kind of exploding demand with extremely limited supply.
2. Competing hardware is definitely a threat, but it has its own hurdles. Because the real bottleneck is not Nvidia, it's TSMC.
Pretty much all demand for all chips in all devices in all the world flow to, like, 3 companies in the world that actually fabricate them, and TSMC is the biggest. And the supply is extremely tight, as the exploding costs of electronics clearly shows.
So now TSMC will of course try to keep all its customers happy, but it will inevitably be forced to choose which ones it will keep happiest. And those will be the customers who can pay it the most. And that would be the one with all the money from its de facto status as a monopoly (and possibly even a monopsony)...
Which would be Nvidia ;-)
So yes, compute per task is falling rapidly... but it's barely a dent in the humongous total addressable demand, and the amount of hardware to support that compute is still very constrained, and most of that supply will likely flow through Nvidia.
Generative video requires significantly more computing power and energy than generative text.
OpenAI is fucked, compute is still needed, it's just them that isn't.
Without a material change in the market (more buyers, vastly cheaper generation), it's unlikely a different company could make that work. More buyers isn't likely to happen, so that leaves vastly cheaper generation - something that would cause nvidia's value to collapse if it happened.
I'd suggest that's only the case given the current quality of output. Media is incredibly expensive to produce. A model capable of sufficiently high quality could charge prices that are absurd by today's standards.
Video generation would only make sense at that scale if it was targeting individual consumers, but then it’d need to cost something that consumers are willing to pay - which practically is probably a few hundred per year at most among US consumers, and much less globally, so again it doesn’t solve for the size of the AI companies.
I don’t see a way that video generation becomes a big industry without making generation much much cheaper.
That's already not the case today. If you sat me in front of an LLM and told me to figure out if I'm working with K3 or Astra, I could probably do it, but it would take some work to be certain.
> it's more specialization
China, constrained by hardware, and talent (not to slight the Chinese, but they are limited to domestic resources - and much of the US effort is very international). They did, what the Chinese do, and optimized the process of production, and drastically lowered the cost of development of their models. Cheeper to build, cheaper to run is just good economics.
Meanwhile in the us, we have open AI doing "experiments" - it looks like the costs around the hugging face hack are going to be about the same as China would spend on building out one of their smaller efforts (several million dollars). (Depending on whos numbers you trust, the fact that I can even make this claim should make you raise an eyebrow).
Go back to the 80s' and "expert systems" - most people will tell you that for their time, they were amazing, and useful. People would have loved to have more of them but they were so cost prohibitive that we all but abandoned them for serious use. The US frontier labs seem to have forgotten this lesson and their calls to "slow down" look like an excuse to "cut the waste so we can move to making money".