"This will never work" is a pretty confident assertion for a field that's so young and rapidly evolving.
This is what the parent said. AGI won't rise out of AlphaZero and AlphaFold in the same way AGI won't rise out of Houdini chess engine. This is the industry consensus.
That's a straw man. Nobody thinks AGI will rise out of domain-specific systems. The question is whether domain-specific systems are necessary for AGI.
Of course, the problem is that AGI isn't a well-defined concept. But if we define it as achieving superhuman performance across several hundred domains where there are objective measures of success, it doesn't seem far-fetched to predict that it will involve some general reasoning system paired with a bunch of specialized modules.
> I don't foresee AGI arising out training bigger LLMs
I agree that AGI will involve tool usage but not only involving domain specific AI models.
But lets try to find the discriminating point in the discussion - do you believe AGI will necessarily involve training bigger LLM's or not?
I believe they are necessary. WBU?
No, I don't think LLMs are necessary for AGI at all. I think there are multiple paths to AGI, some of which involve LLMs and some which don't.
This bet is too early.
> Why didn’t OpenAI release a math specific model? Why not a literature specific one? Why do they instead have generic models of different sizes? And how did all labs converge on this?
Because they have a very early product and they could train it, brute force, with access to an extraordinarily large pool of money. So did all the other labs. Because it was thus easier to scrape everything rather than spend enormous effort (with tools that did not really exist) to partition the training set. Any number of other "because"s.
It's just what they are doing now and it showed the earliest results.
LLMs are still less intelligent than rats, which have tiny brains.
The point I am getting to elliptically is that larger models aren't necessarily the solution. They are one solution pathway.
It is fully possible (I think actually likely) that the ultimately successful path for LLMs will emerge from the pressure of keeping them small, not making them large. Very small, domain specific models could well outperform large models in their domains, and they might even show that domain specialisation is not necessarily much of a limitation, just a useful impetus to stay small. Like how rats can drive those little cars.
(I think the frontier models are potentially already too big. Can't prove it or even close to it, but it feels like this is going to be a story.)
The answer to your question is “because the market isn’t big enough”, not because it doesn’t work. Why would knowing about 2019 internet memes help you in any way at coding?
99.99% of the knowledge an LLM has is useless for a given scenario, the hard part is knowing what the .01% that’s needed is. Knowing as much as it can means the model can handle edge cases, turns of phrase, etc.
Put another way, it avoids overfitting. That’s basically the insight that’s given way to the current AI boom.
https://github.com/Brainrotlang/brainrot
"Brainrot is a meme-inspired programming language that translates common programming keywords into internet slang and meme references."
They did and retracted it because they found that GPT 5.5 beat codex pareto optimally. This keeps happening.
> because the market isn’t big enough
Huuh? market isn't big enough for AGI? The parent suggested that AGI would emerge from this process.