I think the problem with this argument is that it's too inclusive. Is the bacteria that's adapted to an antibiotic conscious? It's showing intelligence right? I think if you're going to say something is potentially conscious, for me to take the argument seriously at least, there needs to be some plausible mechanism. I just don't see one for LLMs.
maybe the bacteria are conscious. How sure are we that they're not?
The only strong argument I have against it is the anthropic principle -- there are billions of times more bacteria than humans, so it's overwhelmingly unlikely that I'd be a human rather than a bacteria.