No need to condescend, I'm very aware of what temperature is for LLMs. But I'm going to push back - if you're claiming all LLMs simply do is a stochastic _search_, how can that produce novelty, in the conceptual sense? (I'm not, for example, talking about novel rearrangement of existing ideas and code)
> We know enough about what an LLM does, and what a brain does, to be quite certain they don't work the same.
I don't think the claim is that LLMs do what brains do - I think the correct form of the counterargument is that _whatever LLMs seem to be doing_ produces end results that were previously only possible through the application of human intelligence, so there must be some axis of however you define human intelligence that LLMs currently seem to display as an emergent behaviour.
By reaching into the voids of its embedding space and returning tokens related to nonexistent semantics. Or, if you like, "hallucinating". The hallucinations which are useful we might call "novel".
> _whatever LLMs seem to be doing_ produces end results that were previously only possible through the application of human intelligence, so there must be some axis of however you define human intelligence that LLMs currently seem to display as an emergent behaviour.
I don't think that has earned its therefore. Another perfectly reasonable explanation is that LLM's output is a close enough facsimile to intelligence that if you allow yourself you can easily be fooled into thinking its intelligent. That's not the same category of thing. It's not an incremental step away from intelligence. It's a whole different animal.
This sounds to me like an admission that LLMs are not just doing a stochastic search, then.
> close enough facsimile to intelligence
What's the distinguishing criteria then? How can you tell the difference?
- Something contained in the data set, not necessarily the same thing for every iteration of a given query
- Something not contained the data set (hallucination), not necessarily the same thing for every iteration of a given query
Does that clear it up?
> What's the distinguishing criteria then? How can you tell the difference?
All the ways they fail to exhibit intelligence. They can't learn. They can't adapt. They can't reason abstractly. They can't count. Etc...
I find the rebuttals pretty convincing - that there seems to be some emergent behaviour that is not simply just next-token-prediction, or that the ability to do accurate next-token-prediction requires something "extra" that LLMs have.
> All the ways they fail to exhibit intelligence
Another implicit admission that there _are_ ways that LLMs exhibit intelligence?
The next step then would be to design and conduct experiments that isolate this effect. Figure out how to make it happen reliably and in such a way that you know it's actually happening as opposed to just something you're imagining. Isolate it or distill it so it can be studied directly. Until then, it's easiest to dismiss it as imaginary.
And you're happy that the replication of LLMs across many foundation model companies is insufficiently reliable?
> just something you're imagining
So the alternative explanation you're suggesting to emergent LLM behaviour is mass independently-corroborated human hallucination. Which is more likely?
Also it really does seem like you've moved the goalposts a lot here without really giving me a substantive response.
This is not at all what I was saying. I think you've already conceded that LLMs demonstrate emergent behaviour but you dismissed it as a "close enough facsimile to intelligence". I was saying that the emergent behaviour is reliably replicable, in response to your following statement:
> Figure out how to make it happen reliably and in such a way that you know it's actually happening as opposed to just something you're imagining.
I think there is real work underway in the area of interpretability. In the meantime, there appears to be plenty of empirical evidence for the claim that LLMs exhibit some sort "intelligence" in the enormous penetration that agentic coding has achieved in software development? Do you deny the usefulness of LLMs here, or are you going to assert that actually software development requires no intelligence of any sort?
No. Please don't put words in my mouth. What I said is that an LLM compresses a bunch of information into a semantic embedding space and then does sort of a stochastic search in that embedding space. Any similarity to "intelligence" is accidental. You may look at the results of that process and "see" thinking or reasoning or something, but it ain't there.
> "intelligence" ... agentic ... usefulness
I don't think LLMs need to be intelligent to be (at least narrowly) useful. No more than random forests or genetic algorithms do at least.
[edit] Look, this has devolved to the point where it's no longer productive to continue. If you're going to state things like this as fact, there's really nothing more I can do here:
> emergent behaviour is reliably replicable
Go collect your Nobel prize then! This is no longer a discussion grounded in reality.
On what grounds? I don't think you've provided any evidence other than LLMs can't "adapt" or "learn" to show that LLMs do not show intelligence in any way. I think it's clear that there must be some emergent form of intelligence over words from just the agentic coding ability alone. I am not claiming that LLMs are intelligent, only that they display aspects of what we understand as intelligence.
> I don't think LLMs need to be intelligent to be (at least narrowly) useful
I agree! But they are more than narrowly useful, and they absolutely do not belong in the same category as random forests or genetic algorithms!
> Go collect your Nobel prize then! This is no longer a discussion grounded in reality
Once again you are being condescending while misrepresenting my position. The emergent aspects of "intelligence" have been replicated by virtue of independent LLM vendors training their own models - I am not making a stronger claim, you have misunderstood me.
Thanks for participating.