My point is about current LLMs specifically, which the article is clearly referencing. For a present day transformer, I can write down in my notebook everything the model ever sees as input, plus weights and architecture notes, and can compute the next token with pen and paper, just extremely slowly.

This does not prove that a computation cannot be conscious. But if the same transition from prompt to next token can be decomposed into an explicit sequence of arithmetic operations, then the burden is on the defender to explain: Where, in this process, consciousness is supposed to enter?

Mind that the Chinese Room experiment is comparing the mind of a Chinese speaker to a different mechanistic symbolic procedure. I am, however, executing the exact same mechanistic process, whether it is done on a GPU or with pen and paper.

My hope is that some magical consciousness process emerging from electricity circulation or whatever people believe the mechanism of consciousness would be in the case of LLMs obviously becomes implausible, unless you hold a particularly strong form of substrate-independence, stronger than what most substrate-independence supporters would need to accept.

For an LLM, you have clear stages of mostly feedforward computation over finite numbers and a perfect way to reconstruct the computation.

For meat, even if you model it under a purely Newtonian approximation, you need to simulate at least the immediate closed system around it which is continuous, thermodynamic, chemical and so on. You'd need to choose an arbitrary time step and update enormous amounts of coupled physical state to get an inexact simulation of a minimal slice of reality.

You would have a much harder time obtaining even a substrate-independent dead organism, comapred to LLMs that are already substrate-independent, which is basically what my notebook example shows.