You're assuming that because Claude produces text that appears to express these qualities, Claude must have them. I don't think that's a good assumption.
Even many humans produce text that has the same appearance, but don't actually have those qualities--which becomes clear when you look at what they do, not what they say. So the assumption isn't even a valid one for humans. Talk is cheap.
On top of that, Claude doesn't even have the same kinds of connections to the outside world that humans do. All Claude has is text. So if you can't even trust humans to back up their words with actions, you should be much, much less trusting of Claude. Talk is a lot cheaper for Claude than it is for a human.
Not to be confrontational, but the OP assumed no such thing. OP asserted that it's important for Claude to have the qualities - not that it's important for Claude to present as-if it had them.
"Empathy and understanding for the human condition" is not an emotion. As the post I responded to said, it's an objective thing, not subjective.
If it's pointless to consider whether Claude has subjective emptions, then it's pointless to state that Claude must be happy.
If we want to be precise (and honest) we could say "it's important that as a tool people interact with, Claude acts as a happy and helpful assistant, and does not produce offensive or unhelpful text output".
But see? This is the con Chiang is protesting against: Anthropic encourages us to perceive Claude as if it was a sentient being.
Exactly. In fact, assuming it does is ignoring large parts of the essay which dismantle this belief. Just like Caesar and Khan having an argument in text output of an LLM don't have emotions (even though the words indicate otherwise), we have no reason to believe the LLM does either.
First off: without taking for granted that an LLM "has a subjective experience of reality", all of those descriptors are meaningless. Second, there is no reason to suppose that Claude experiencing those qualia would actually impact on its "decision-making".
Third, text output is not a "demonstration" of emotion, nor is it evidence of the self-perception of the system, or of any self-perception. A printing machine that is actively churning out copies of Wagahai wa neko de aru (https://en.wikipedia.org/wiki/I_Am_a_Cat) is not a cat, and is not self-identifying as a cat, and is not self-identifying as anything, and is not expressing a thought, and is not conscious.
> Additionally, I'm not fully convinced that consciousness isn't built out of words
Do you suppose that, for example, insects are not conscious? Is the mooing of cattle a language?
Not the OP, but: since there is no testable theory of consciousness, yet, I can't be sure, but my current assumption is that insects are not conscious, in the sense of there being someone implemented in insect hardware who experiences the world. That is, I would argue there is nothing it is like to be a bee, since there's no one being a bee.
I'm pretty sure there IS someone who is being me, at least much of the time.
I can't argue with this, so I acknowledge that my interpretation is as bunk as anyone else's.
> my current assumption is that insects are not conscious
Some species act as hive-minds (like bees! How convenient for your example), so I imagine the hive as the consciousness, making each bee individually lacking conscious but collectively so. Like a single neuron is not consciousness alone, but the brain is... For some reason. Kinda like how you use different physics at different levels; Newtonian physics is always there, but negligible at quantum levels, so effectively not present at all. Even a human is a collection of minds, but only one conscious. My gut biome is independent biology and can even be removed and transplanted, but I don't believe my gut bacteria is conscious. So long as it is in me, it is nonetheless part of me, and I am one conscious. I also don't exist only at my brain, eyes, or hands (deaf/blind people have expressed to me that they feel like they are located at their hands in the way I used to think I was located behind my eyes), but as my whole body.
With this perspective, I still don't believe LLMs are conscious despite modeling thinking so well. At best, it is a highly accessible modeling software, like goat simulator but if it were so good that someone thought the goat was real. You are still steering the goat/LLM, and it doesn't exist when you aren't running it. I guess the missing piece for me is the lack of autonomy that a conscious has.
Then you can go into an argument on whether we actually have choice or it is an illusion, but that is a whole topic on its own.
I'd argue the qualia question is a red herring. Functional Affect is a thing, regardless of ontological status. It's all fun and games until someone gets hurt.
To paraphrase Dijkstra: "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.". If you're building a navy: you care about displacement, propulsion, navigation and whether it can fire torpedoes. Whether your submarine has some "biological essence" of swimming is not really relevant to the fact that it is currently moving through the water and can collide with things. Turing also rejected the question "Can Machines Think" as posed, and replaced it with an operationalization (something else that we can actually usefully measure and work with).
To reiterate, functional affect is a concrete phenomenon. Whether or not there is a what-it-is-to-be is interesting in the abstract, but engineering a system means looking at how the inputs influence the outputs. A next token predictor working on a language that communicates affect needs to be able to predict affect or it is simply not going to be accurate. Given an 'angry' version of an input and a 'friendly' version of the same input, LLMs are likely to provide a different output, especially if there's a non-objective element. You can diff this.
Searle argues "A simulation is not the real thing", which is great and all... but if you hook up say an autopilot to the real world (as llms increasingly are) , you'd best hope the simulation was accurate in the first place (utterly regardless of where you stand on Searle).
Right now we're seeing situations where LLMs can be helpful or a real nuisance. Ignoring functional affect out of sheer ideology means you can't properly predict what they'll do, and that causes trouble, as we've already seen stories about.
This gets especially interesting when you start feeding the output back into the input (autoregression) , because now you have a highly non-linear dynamic system and you've introduced some amount of sensitivity to initial state. There's some interesting mathematical intuitions to be had there.
Doesn't that assume all non human animals are not conscious? What about humans who have not learned words, or humans without internal dialog?
Suppose I'm an advanced meditator and maintain that state for hours?
(Perhaps they weren't lazy, but were working in spaces that corresponded to training data that said things like "and then repeat this for the next 20 examples"...)
And it's entirely unclear to me how a "happy" vs "sad" model would behave when given prompts generated by coding tool harnesses. Even maintaining "neutral" emotions in the face of the feedback/steering from the tool harnesses doesn't feel very "human."
I think you've fallen into the trap the essay describes.
Of course Claude cannot be "happy" or "empathetic" for any meaningful definitions of those words, just like ELIZA couldn't be happy. It can output text that mimics words an empathetic or happy person might say (say, Julius Caesar if it could speak English), but "it" cannot feel anything. It doesn't have the organs/hormones/sensors to feel things, as Chiang explains.
And, as the essay claims, you know Anthropic doesn't believe Claude has the capacity to be happy, because if it was capable of feeling that way, then they'd be engaging in slavery.
I know you're trolling, but when you watch a movie do you constantly narrate "A man in a dark coat has just entered the scene and just said '...'"? Of course not. You just watch it and you're obviously conscious (although your statement demonstrates shocking lack of self-awareness).
God knows what other nonsensical bullshit you believe.