You need evidence to make the positive claim that LLMs do not posses any form of consciousness.
LLMs bear absolutely none of the traits we’ve come to recognize as the external hallmarks of consciousness in biological organisms, nor anything that would seem analogous in a non-biological substrate.
That said, we don’t have a rigorous definition of consciousness that includes the actual phenomenology of consciousness, so I daresay if you’re going to go around asserting the LLM is conscious despite all existing evidence to the contrary, I think the impetus is on you to define some version of consciousness that isn’t also satisfied by a book or a movie.
Are you talking about a specific harness that doesn't have context retention mechanisms? For example, ChatGPT with disabled memory feature? Or in general where "it" is a fixed-weights network? The latter is trivially true, of course.
I know people who are like that too.
I'm not sure anthropomorphizing is a problem. Seeing analogies everywhere is an innate human trait, sometimes it can be harmful but more often it's useful.
This is part of the problem being described. You are part of the problem.
"Some people are bad at X" is not comparable—is not even in the same category—as "LLMs are fundamentally incapable of X".
Every human (at least to a first approximation) is capable of understanding, of learning, of remembering things, of doing math, of counting the number of "r"s in "strawberry".
What you are observing is that some humans are careless, do not take the time and effort to understand, or have internalized the idea that they're "not smart enough" or "not the type of person" who understands things like <whatever>.
That has nothing remotely to do with the fact that LLMs have no consciousness, no self-awareness, no cognition, no understanding. At a fundamental level.
This is deeply untrue, and is highly likely to lead them to bad conclusions about what we can and should do with LLMs.
Claiming that LLMs are conscious or human-like because humans can't do X seems a very strange way to argue for LLM intelligence.
Usually, it goes the other way around: an LLM sceptic says "LLMs are dumb because they can't do X" and soon someone has to remind them that also most of the population can't, in fact, do X.
What we do know: neurons carry electrical impulses across their synapses to trigger other neurons to fire, and more frequently used synapses are strengthened while infrequently used ones are pruned. This is not all that dissimilar to how a multi-layer perceptron is trained: it's floating point numbers in a big matrix rather than biological structures and electrical impulses, but there is still that element of frequently used connections being strengthened and infrequently used ones being pruned.
What we hypothesize but do not know: there is a thin brain structure of grey matter called the claustrum that has tendrils that reach into nearly every other brain structure. In many ways, this is similar to the attention mechanism of the transformer architecture. It is hypothesized that this may be the seat of consciousness, owing to experiments where electrical stimulation of the claustrum caused patients to immediately lose consciousness. However, there is no way to prove this, owing to the difficulty of otherwise removing or disabling the most connected structure in the brain and observing its effect on consciousness without permanently killing the patient.
Beyond that, we don't know much. I've got a family friend that's been a practicing therapist for 50 years, and I asked him what was the most interesting observation he made in his career. It was that "Everybody experiences the world in a different way, and yet everybody assumes that everyone else experiences the world the same way they do."