"Yet if we build systems that can suffer, and miss their sentience, we risk creating suffering on an industrial scale."

--------------

Will we recognize AGI when it's finally made? Will we go from something that is not conscious to something that clearly is in one clean step, or will we murder millions of artificial lives while iteratively developing from the former to the latter? Will the tests we develop even work, or will AGI be alien enough that it takes us time (and a lot of murder) to recognize it for what it is?

I personally don't think LLM's are capable of consciousness, but I also can't prove they aren't.

I think Blaise is on to something, but you have to shift your definition of conscious toward conscientious for it to make sense.