However, if you had had demanded someone for a blueprint in 1925 of how to design such a magic bullet, especially a magic bullet that targeted virtually all forms of bacteria, it would have sounded ludicrous. Yet, 20 years later, the world was manufacturing 6-7 trillion units of penicillin a year, capable of treating 3-6 million people. And that’s in spite of the fact that Fleming’s work sat mostly untouched for a decade before Howard Florey and Ernst Chain seriously set about to isolate and purify the substance.
You can quibble and say that penicillin was discovered, not designed, which is certainly true. But I would ask you to consider, does current AI development look more like design or discovery? Does it look more like analytical engineering or evolutionary selection? I would say on both counts the latter, in which case, we should prepare to be surprised how long it might take to make revolutionary advances. And that’s on both sides of the ledger, we might find ourselves stuck in the current paradigm for a long time. But, we might not be.
I wouldn't bet on evolving an intelligent, sentient being-in-a-box on a computer any time soon though. I'm of course prepared to be pleasantly surprised.
That said, I think it's pretty clear that LLMs are not going to get us there.
This is obviously complicated by the fact that LLMs/Agents are useful by themselves, but that’s not really the topic at hand.
This is why biological comparisons are weak, we talk about a few agents verifying and checking LLMs, meanwhile the world consists of almost an infinite number of the same, just operating on different time scales. I agree that with we don’t know the timescale, and we definitely don’t know if long term it will continue to work ”adding more of the same”. Throwing more penicillin at the problem sure as hell didn’t, but it looked great initially. And I’m obviously not arguing the human benefits of penicillin, just that what we thought would work forever quickly didn’t.
> Life, uh, finds a way.
Imagining something in advance is not necessary at all for scientific advancement. This is particularily true in AI, and no one expects to imagine what superintelligence is until after it is created. You set up your datasets, your architecture tweaks, and measure the results on some set of benchmarks. There never was a blueprint, no plan beyond the experiment itself. We're not even close to understanding the things we have already created, and yet we created them. So why expect anything else for the next step?
That is simply not accurate. There are examples of scifi novels, novellas and other media that dealt with it. We can argue over whether it was that exact format, implementation and so on, but that 'shape' ( to use a common llm term ) of technological advances was very much explored.
Then why does anyone expect to create it? I'll take a stab at an answer: they think an LLM is some kind of "incremental improvement" and therefore a step along the inevitable path to discovering AI. But that seems delusional to me. I can't imagine anyone sound of mind who knows how an LLM works thinks it's actually intelligent. So in what sense is it an "advancement" on the path to AI?
The concept of an incremental improvement in an objectiveless search in a high dimensional space is.. absurd.
It's reasonable to doubt that LLMs are a path to AGI, but I don't understand how this is still a matter of dispute in 2026. What's your definition of intelligence that doesn't cover an entity that can translate fluently between dozens of languages and also solve open problems in mathematics? And be real-if you have one, is it a definition you or anyone would have given a decade ago, or are we doing "god of the gaps"?
That's more or less looking for interesting patterns in a jpeg or another lossy compression result. It's interesting that the models seem to be able to (fairly) reliably return relevant chunks of the image. Even more interestingly, they seem to be able to invent plausible chunks of image that aren't even there. That doesn't meet my bar for intelligence though. I'd need to see it learn and adapt. I'd need to see it be clever, not merely "knowledgeable". I'd need to see it capably analyze itself. I'd need to see it reasonably estimate uncertainty and know itself in the sense that it has some idea how right or wrong it is about something. I'd need to see it exercise judgment.
I don't think I'd give a different answer a decade ago but who knows.
[edit] For all we know, one of the salient features of intelligence is that intelligent beings are incapable of precisely defining it. I'm not sure how productive it is to attempt to do so.
Actually, stronger - it's valid in some circumstances to say something is infeasible to precisely to define and you'll just know it when you see it. But I don't think it's reasonable to take that stance and then assert that "anyone sound of mind who knows how an LLM works" must agree with what you see. You gotta pick between striving for rigor and denying your opponents' soundness of mind.