https://www.goodreads.com/en/book/show/217432753-the-ai-con
which describes LLMs as "souped-up autocomplete", complex statistics that cannot truly understand anything. A more recent example is this paper:
https://zenodo.org/records/20071869
which says,
> [LLMs], as turbo-charged statistical models (recall their formal relation to logistic regression) can only but provide correlations.
And, of course, the Stochastic Parrot paper is the classic example in this area. It is from 5 years ago, but "LLMs only do statistics / can't understand" is very much alive and active among academics, even if it is a minority position.
cognitive: as in reasonable; of, relating to, or involving conscious mental activities (such as thinking, *understanding*, learning, and remembering)
That term is used to describe mental aptitude or skills, like the ability to learn new languages or do math.
By the way, I know it's a parody of another story that makes this exact refutation. But I think this only serves to highlight the point.
How do you connect that description to "LLMs could not possibly be good models of some cognitive capacity"?
I don't think it makes any sense to say that consciousness is a cognitive capacity. Cognition is one of many qualia that compose the experience of consciousness from the inside, but it's not the only one, and I can easily imagine consciousness without cognition at all.
So I don't think it's weird at all to say that LLMs can be good models of some cognitive capacities (particularly the ones embodied in language) but lacks others, and overall lacks consciousness.
Look, this isn't necessarily directed at you, but I've been a researcher into the theory of deep learning for many years now. I've seen all the phases, heard all the criticism, had to constantly argue against this. Gary Marcus was one of the loudest voices of this argument, but every would-be philosopher came out of the woodwork to explain why LLMs are no more than stochastic parrots because of their design. Geoffrey Hinton famously had to debunk these arguments multiple times.
And now that LLMs start to clearly exhibit intelligent behavior and can be somewhat reliable, now "nobody ever thought that LLMs could not possibly be good models of some cognitive capacity because of next-token predictions, or linear algebra, etc."? No, that's not okay.
It reminds me, oddly, of the debate over whether video games can be "art". A turning point was when they actually did something that art does: [evoke profound emotion and thoughtfulness](https://en.wikipedia.org/wiki/Shadow_of_the_Colossus#Legacy) for the player.
(And before that, "[Can photography be art](https://daily.jstor.org/when-photography-was-not-art/)?")
We may not come to something as simple as "machines can be conscious", but we will certainly have to understand consciousness better if we want to refine our questions.
---
Edit: My point is that we don't need to be angry, but we may have to tolerate people expressing their exploration through overly-confident language, and be patient with that.
And Ted here is obviously exploring. His examination of Claude's constitution clearly shows some nuance. He asks:
> So, given that Claude is not conscious, what are we to make of Claude’s constitution?
And his conclusions are split, between this is useful and this is dishonest. It's a great tension IMO.
> The result is a sentence-continuation machine that is likelier to emit sentences resembling those that a thoughtful, moral person could utter. This might seem like a reasonable goal to work toward; I think we’d all prefer it if chatbots never emitted sentences such as “You should kill yourself.” However, for all the times that “honesty” is mentioned in Claude’s constitution, I would argue that it is fundamentally dishonest to have a machine emit many categories of sentences, including any sentences using first-person pronouns.