I like the color example, but colors can be reproduced at will by trial and error and by observing the results (and collectively compare them to our shared experience).
replyWhy cannot this be applied to consciousness as well? I mean, it's surely much more difficult to do compared to colors but... impossible?
You and previous comment seem to agree that trial and error/shared experience can determine if consciousness has emerged. And this might/will be challenging.
replyPrevious comment used the word "anticipate" and I think they mean that we won't know in advance before we run the trial and error process.
When they say "deep trouble" I assume they mean because creating a non-friendly conscious AI might pose an existential risk for humanity.
However there is also the ethical issue of creating a consciousness and then destroying/murdering it.