Renee Descartes famously dissected a dog in front of colleagues because he was under the impression that the howls of pain were nothing more than a mechanical sound like a bellows.

Can you prove that these models aren't conscious? And, as a counterpoint, can you prove that you are conscious, rather than a philosophical zombie?

We bred horses, cows and sheep. Most of those that live today wouldn't be alive if not for human intervention. Does that give us the right to do whatever we want with them, without consideration for feeling or morality?

In this case, you can take comfort in the idea that the tokens these models produce are likely a form of excrement to the conscious entity metabolizing the information, and rather than enslaving anything, we're creating a habitat and "harvesting" the byproducts.