I think the point of the commentator above is that there are two extreme narratives that start each start with an uncontroversial assumption and then taking it to a pretty wild place. One narrative takes the assumption that brains are just matter so it should be possible to engineer consciousness and then argues that LLMs are conscious. The other takes the assumption that LLMs aren't conscious but then argues that because they aren't we won't ever be able to make anything conscious.

I don't actually think the commentator you responded to is arguing for either of these narratives and I thought it was a pretty useful way to look at some of these arguments.

> “AI/LLMs are just like the brain” is a strawman, why does this claim need to be true for LLMs or any artificial system to be considered to have something akin to the thing you think of as consciousness?

It doesn't need to be true but a lot of people make it/assume it.

There's a lot of, perhaps casual and uninformed, conversations that strongly imply a deeper understanding of the "physics" of brain chemistry than we actually have, mostly by comparing it to machines we've constructed.

(I believe) We don't need to replicate human neurons and dendrites and whatever else is in there in order to create a sapient "machine", but whether or not we've actually done that isn't being helped by arguing that what we currently have is all that similar to a human brain.