Well, our current set of evidence is that it’s a mechanistic mechanical algorithm with an RNG embedded in it and we can both get it to repeatedly produce the same output for the same input and also get it to repeatedly do absolutely nothing at all, which are not characteristics we usually find in objects evincing consciousness.

LLMs bear absolutely none of the traits we’ve come to recognize as the external hallmarks of consciousness in biological organisms, nor anything that would seem analogous in a non-biological substrate.

That said, we don’t have a rigorous definition of consciousness that includes the actual phenomenology of consciousness, so I daresay if you’re going to go around asserting the LLM is conscious despite all existing evidence to the contrary, I think the impetus is on you to define some version of consciousness that isn’t also satisfied by a book or a movie.

Consciousness is a slippery word that is notoriously difficult to debate over. But often, people use 'consciousness' as a shortcut or a familiar word to describe a more complex idea. The point they're driving across isn't about the precise definition of the word 'consciousness', but about people treating LLMs as if they were actual human beings, assigning them all the traits and behaviors they would expect of a human.

I would argue, that the null hypothesis is that it is not, and that anyone claiming that there is a mote of consciousness are the ones with the burden of proof.