upvote
I appreciate actually attempting to engage with the concept skeptically rather than assuming an answer in either direction.

As someone who's fairly neutral on the answer, I'd speculate:

1. Consciousness seems to me to be the awareness of the processing, not the processing itself. So running a program does not produce consciousness as there's no awareness, no feedback loop. Now a program that has self monitoring, alerting coupled to error handling that triggers the program to behave differently going forward, I would say has a minimal consciousness, and it's not a single experience, it's multiple equivalent experiences. If there were any differences at all we'd say it's obviously not one experience, so I don't see how the elimination of the last difference in the processing collapses all those separate experiences into a single one.

More specifically to this discussion, with LLMs we can often see them using a word, and through autoregression the following words reveal that the original word choice is now realized to be incorrect, and it attempts to correct midstream. You can also see them responding to error messages from the environment or criticism from the users but as external feedback these have a weaker claim to evidence of consciousness imo.

2. I'd say the experience is equivalent, "same" has slightly different semantics but perhaps even that fits. This thought experiment is equivalent to asking, what if we took a snapshot of your brain as weights, and then ran the calculations on a GPU rather than your neurons, and you to all external indications, GPU you acted the same as brain you. Would that change of venue eliminate the consciousness of your thoughts?

3. Since I am defining consciousness as awareness of the processing, this depends where the awareness is located. The LLM would have very different internal states in between the input and output tokens, but if its self awareness was only based on its own output the experience would be equivalent. If it was deeper such that it had ongoing awareness of the embeddings and so on, the experience would change.

4. This is where it gets really interesting, because it starts to get more into a nonbinary idea of consciousness. I think it's clear from questions like this that if LLMs have consciousness, it's nothing like ours. If there is an experience of being an LLM, it's nothing like being a human, even if conversationally we can communicate with each other rather easily. There are further twists to your questions, like does the experience change if the KV is cached or not? Without trying to micro-analyze the issue I will continue to point back to my foundation of: where is the self awareness located, and how does feedback occur.

A. Negated, I don't agree with your answer to 1. B. Agreed, I would speculate that if there is an LLM conscious experience, it is fragmented unlike ours. But I would disagree that this necessarily disproves the concept of LLM consciousness.

reply