In short, it's the "mind's 'I'" - we think not as response to external stimuli (only) such as prompts, but we have an inner "I" that asks questions on its own initiative. There are people like Douglas R. Hofstadter, who believe consciousness is not linked to human hardware (the brain), but that it is an epiphenomenon that emerges as a result of sufficient complexity of the underlying system: https://en.wikipedia.org/wiki/The_Mind%27s_I

I believe that while underlying high complexity is certainly logically necessary for consciousness, but it is not logically sufficient, and I am undecided (slightly "pro" intuitively) on the question of separability of consciousness from its hardware.

Will a LLM ask an original question on day? I doubt it.

Note that AI models do not have to be conscious to be useful (or to take away millions of jobs)!

I don't think that quote from the article is disagreeing with you at all. Like you said, we don't have a cohesive definition or test of consciousness, so research like this doesn't say anything about if this is or isn't similar to human consciousness.

I would guess Anthropic included that sentence to make it very clear they're not claiming human-like consciousness, and dampen journalists writing headlines like "Anthropic discovers their AI thinks just like humans and may be conscious".

edit: later in the article they even more explicitly agree with you

> Our experiments don't show Claude can have experiences, or feel things in the way humans do—in fact, it’s unclear whether any scientific experiment could prove this to be true or false