AI is essentially infinitely reproducible at zero cost, and won't suffer from decay etc. There's not scarcity to preserve.
So, I'd turn off an AI in a moment to save property or real possessions or money. I'd sacrifice property and money to save animals. I would never choose to save an animal over a person. I'd probably not choose to save a person over a child.
I don't see any inversion of any of those priorities that makes any sense.
It is interesting to think about what would cause me to consider these priorities incorrect, but a majority consensus about a program being sentient isn't it.
Training AI is often more costly than supporting human from birth till death. Just sustaining frontier LLM model on necessary hardware costs more than living in first world countries.
What if genetic memory, multigenerational conditioning, life-long patterning and conditioning, experienced in a body, combined with forces and processes not yet detected nor explained, cannot quite fit in a sliver of modeling?
"We must not consider consciousness as all too important because what matters is human flourishing and human rights."
And some of
"Even if we suddenly all agree an LLM is conscious it wouldn't and shouldn't influence us very much"
While acknowledging that some people will change their lives, the way some people like myself won't eat octopus or apes because it is probably more like murdering a sentient creature.
And oh by the way you cannot murder a copyable individual anyway. Did you read Enders Game?
>And oh by the way you cannot murder a copyable individual anyway.
I think for many people the concern is not so much about violating the desire to live that a conscious software entity might (or might not!) have, but about the subjective experience of suffering it might undergo.
I hope the same becomes true of people, and that doesn't mean people stop being sapient.
I think you should reconsider this viewpoint. Suppose that we really can create silicon-based consciousness, in that case your view would result in a huge amount of suffering.
Take some other basis for dismissing digital consciousness, this one is too dangerous.
I argue zero - placing AI below the value of humans no matter the energy input.
The _only_ reason an AI might be worth saving is if it, say, has a cure for all diseases, but then we're not saving it due to its intrinsic worth, we're saving it because we can save many humans. I _would_ consider the trolly problem a legitimate thing in ethics, but not if an AI were tied up on the tracks no matter how expensive it is. It's a thing. It gets run over to save any human.
What would a silicon-based consciousness desire to cause suffering?
If not, then your comment's claim is false.
Anyway, the deeper solution is to acknowledge that all life is sacred, and infinities cannot be compared, and some decisions are impossible to make, and some tragedies cannot be averted, and "prioritization" is a distraction that forces choices when choices are not strictly necessary.
The pure rationalist loses something important.
> infinities cannot be compared
That's either a mathematically illiterate assumption or a very strange philosophical hill to die on.
> some tragedies cannot be averted
Sure. The question is what to do about the ones that can be averted.
> some decisions are impossible to make
> and "prioritization" is a distraction that forces choices when choices are not strictly necessary.
Again, the question is what choices to make when you can (arguably must) make them. Saying they're impossible is just refusing to take responsibility. You either do something, or you don't.
If you believe sanctity of all life is a solution, then I'm curious what you believe the problem is that such a belief solves.
I bet it's circularly defined as justifying the preservation of sacred nonhuman life? I'm not trying to be provocative just curious.
They barely touch on the issues of consciousness, Picard basically says "What if Data is conscious?" and then goes off on a tangent. The judge eventually rules in Data's favor but doesn't give much of a justification IMO.
It's still a good episode, but it doesn't add much to the conversation on consciousness. It's a hugely complicated topic which people have devoted their entire careers to.
Does the ship's computer have a commission?
It was a good episode but it had some elements of Star Trek tropes in it, like the evil admirals and Picard can talk his way out of anything.
Data is basically an Isaac Asimov android (down to the positronic brain) and Measure of a Man is an Asimov-type story whose tropes don't entirely fit within in the Trek universe.
It makes no sense within the context of the Trek universe that Data is unable to use contractions for instance - but it makes sense in the context of how a robot might have been conceived of in the 1940s.
There are also a few early episodes of Voyager where the crew treat the doctor badly.
It seems odd in a universe where these people are having relationships and children with aliens from another planet, that they'd be weird about computerized people.
My retconn is that there must have been a lot of stochastic parrot/AI psychosis in the Star Trek universe when they first started making Majel Barrett-voice computers.
Maybe lots of people got confused and thought they were talking to a person when they started having conversations with the computer, and this lead to an over-correction where people were highly disposed to say "this machine isn't a person" even when it presents like one.
TNG/DS9/VOY/ENT and the movies were all fairly explicitly in the same universe and with characters referencing things that happen in other series and a reasonable effort at continuity.
A example of this is the ENT episode where they meet the Ferengi. It had been established as cannon already that Picard made first contact with the Ferengi on USS Stargazer like 150 years later, so they just added this scene where the ENT characters memories of the event was wiped, so that the "official first contact" could still be Stargazer.
The Neutrek stuff isn't in the TNG universe for a variety of reasons, although I think part of it is they didn't want to put the work in to maintaing continuity. None of the TNG era producers or show runners were involved and nobody they wanted to hire really knew the lore.
That being said I don't think there's really a continuity problem with Mudd's androids because he didn't make them they were from another galaxy, and the fembots weren't really implied to be sentient except when they had a human brain put in them.
TNG kept "accidentally" making sentient machines from Data/Lore to the drilling robots, the nanites, and Professor Moriarty. It does seem strange after all that they still considered Data unique.
TL;DR Picard's initial arguments are pretty weak, even admitting that Riker as opposing counsel almost had him convinced. During a recess Picard talks to Guinan where she alludes to the future subjugation of many Datas which Picard connects to slavery. Back in the courtroom Picard calls Maddox as a hostile witness and gets him to define sentience--intelligence, self-awareness, consciousness--then walks him into conceding Data meets the first two. Picard's closing boils down to, "we don't know if he meets the third--you can call Data a toaster and rule he is property--_but what if you're wrong_". The judge rules on the basis of erroring on the side of caution due to that uncertainty. It's really a great scene.
We're not there yet, obviously. No LLM brings Data's level of awareness but it's as relevant a story as ever because it isn't really about AI but othering for the purpose of subjugation.
A little piece of in-universe lore for anyone unaware or who had forgotten: At this point in the series, Data's positronic brain is new technology no one understands. His creator is missing/presumed dead, the "positronic" basis isn't how Federation technology works, and apparently so far they hadn't done a whole lot of direct experimentation (hence why the trial is happening). So not knowing if he's conscious is a lot more reasonable a stance than with real-life LLMs where we do know roughly how they work.
Also later in the series we meet a character whose brain was copied into a positronic brain, and does imply that technology is at least capable of consciousness, whether or not it applies in Data's case.
I would argue that is a strength, rather than a weakness. Consciousness is unobservable in any entity other than the observer, and its existence in others is pure conjecture, and irreducibly so.
Making it a criteria in a decision involves either acting on fantasy, or, more likely, acting on some unstated basis and using “consciousness” as a dishonest (perhaps to oneself most of all) rationalization.
Debating AI consciousness a real modern equivalent of the cliché (but purely fictional, invented later as a form of hostile mockery grounded in large part in sectarian bigotry) medieval scholastic debates over how many angels could dance on the head of a pin.
When you read about the theological questions that led Christians to kill and excommunicate one another, "angels dancing on the head of a pin" is not far off. The Homoousion, Monophysitism, the Filioque controversy... It's all so arcane and poorly defined. It almost makes one wish Positivism had been invented 2000 years earlier.
The current AI debate about consciousness does remind me of that in one respect: no one can even clearly define what consciousness is.
ST:TNG writing is generally like this. The show required considerable suspension of disbelief and a willingness to accept the kayfabe that deep concepts are being presented when for the most part they're just not that deep. (But it can be very enjoyable when you make those accommodations.)
Note that the episode is NOT about the ship’s computer. They all know it’s not conscious, despite also being a machine that can converse and do things.
Our LLMs are like the ship’s computer.
Because treating things that act human inhumanely is not something I want to learn how to do.
- I'll try not to swear at/hit a printer: not because I see the printer as having human-like qualities of being capable but complex and unreliable, but because I want to be a person who can control his temper.
- Treating an inhuman thing as human because it can mimic us in some way is not something that I want to do.
I try not to make errors like that.
I was bright-eyed and excited about tech once. Like back in 1982 when I got my first home computer and thought CPUs were part magic. Now I know how machines work from the transistor level up to neural nets. There's nothing magical about it. And no consciousness.
Having seen the mockery that the finance-bros have made of "pure tech" (i.e. Jobs instead of Woz, Ellison instead of Joy, etc) and all the enshittification just for pure $$$, I'm leery of ANYTHING ANY tech company tells me anymore.
Now, do I believe that possibly "consciousness" is some kind of state of a super-circuit (our brains)? Sure. Can we emulate that on a computer? We can't even emulate a pebble on a computer (not simulate, emulate). We can SIMULATE what we THINK brains are, but we can't emulate a real one. Not even close, not for many decades.
So in the meantime, I'm going to err on the side of caution.
You do you.
I am not even slightly religious, but they would be abomination.
Or you should.
EDIT: It's difficult to have a conversation when one person changes what they said after the fact.
inhumane: without compassion for misery or suffering; cruel
cruel: willfully causing pain or suffering to others, or feeling no concern about it
You cannot treat an LLM inhumanely, definitionally.
Anyways, when one swears at someone it's typically meant to berate or belittle that person - to inflict some sort of emotional pain. That's the sense I intended when using the word, which is why it fits as a response to what you're saying, and why I would say "don't be nasty to a LLM" has little to do with the LLM itself.
You have a nice day.
People get used to treating human-like, human emulating machines with either disrespect or in a command/control/master fashion, because that's the nature of the tooling.
And then potentially by extent/blurring of lines they then treat other people like machines.
Which is already a thing people do to other people.
I just fear it gets worse.
I think about this quote often, straight from Data's voice module in another episode:
'The most elementary and valuable statement in science, the beginning of wisdom, is, "I do not know".'
The judge broached on the subject of what makes us distinct from Data (e.g. machines w/great heuristics) - the existence of a soul. Or rather, I'd like to think, in the words of CS Lewis, that we are a soul with bodies attached.
A silicon alien coming to earth might poke us, we would say ouch, and just determine the ouch sound is just the result of a bunch of chemistry - not really conscious or feeling pain like it can, just emulating.
I believe that the moment an artificial inteligence is going to "receive" a soul, is the moment it is going to be made to sustain itself. Either as a larger package (some bots working to keep an AI farm running) or as an individual (a bot which is tasked with not only fulfilling human desires, but also sustaining itself)
A similar episode, actually informed by what we know or can forsee about AI and LLMs, and addressing our hopes and fears about what they mean would be interesting though.
So was most of Roddenberry, Piller, et al., Star Trek. At its low ebb in Braga Star Trek, but even then...