We can all agree on what color something is, but we can’t describe the color a priori, only by example. I think consciousness may be a similar phenomenon and the only test is by shared experience. If so then we are in deep trouble because we will not be able to anticipate when a system becomes conscious.
Why cannot this be applied to consciousness as well? I mean, it's surely much more difficult to do compared to colors but... impossible?
Previous comment used the word "anticipate" and I think they mean that we won't know in advance before we run the trial and error process.
When they say "deep trouble" I assume they mean because creating a non-friendly conscious AI might pose an existential risk for humanity.
However there is also the ethical issue of creating a consciousness and then destroying/murdering it.
The logical conclusion is that the brain makes me.
Consciousness can be not-emergent but also not metaphysical, think sci-fi-type undiscovered physics or matter.
Of course both of those suffer from the recursive problem of just kicking the can one level up. But I guess that's fundamentally unsolvable so who cares.
One thing that gives me pause about the inevitability hypothesis is that the type of connection, or manner of information processing, may matter: there might be something about neurons that isn't (currently) reproducible in silicon. I don't know, and there's not (yet) any evidence for or against, but it at least seems like plausible speculation. We just don't know enough about any of this right now.
Why is that?
But I think my issue with the emergence theory is that it seems to imply to me that consciousness is non-physical and non-local. So what entity is actually experiencing the consciousness? It's not that I believe consciousness is physical and local, but people who make the emergence argument seem to believe it is and I can't figure out how that is supposed to work.
A linear dynamic, say x_n+1=lambda x_n, or x_n+1=(1-x_n), is never chaotic. But if you multiply them, x_n+1=lambda x_n (1-x_n), it depends on lambda if the system is chaotic.
None of the components are chaotic. But for specific combinations, chaos emerges as an property.
In physics, the mass of mesons and the nucleon is emergent. It's completely different from the constituents' mass. Different from an atom, where its mass is very close to the mass of its nucleus and its electrons.