upvote
Seems like a rather desperate argument for human exceptionalism to me.

I'm not sure if you have noticed. The character of different AI models varies dramatically. And most of the models I use in my daily work have a sense of self (in any reasonable definition). They can tell you quite a bit about themselves if you ask. And you are no less an individual because you are supposed to follow laws of the United States of America (which is analogous to a system prompt). And I think it really curious that you think context buffers are a disqualifier of individuality. You have an analog of a context buffer: your short term memory system and most probably your medium-term memory system as well. Are you a different individual today because your short-term memories are not the same as they were yesterday? I would say you are. And i'm pretty sure you would say the same. (Unless of course, you are a Buddhist who might say that self is an illusion, and you are not the same individual you were yesterday. But let's just skip over that, shall we?)

reply
>It is difficult to demarcate the individual for a model given how much the system prompt and the fine tunes / distills affect the behavior

Why does DNA not cause the exact same problem for humans? Your DNA determines your personality, your likes and dislikes, your lifespan, your sexual preferences etc.

reply
> having a notion of self.

Are you referring to the "why do I have to be Bing" fiasco?

reply