When a LLM is tired and lazy, how does it recharge and regain motivation?
Humans... sleep or drink some coffee.
LLMs.... idk, you prompt it to try harder? You prompt it to be less tired?
This is what I mean when I say extrapolating LLM behavior based on human behavior is cute.. but usually not useful.
What would be in the statistics? If you go look at your long conversations, working with another, it will be fairly obvious. Keep in mind we're talking next word prediction based on context, not actual action (the LLM doesn't need real rest).
If you went and looked, you'll probably see something like "Great work! Have a good weekend! We can get back to this on Monday." then, next message you instantaneously send something like, "Hope you had a great weekend, let's do this!" and now you're in a latent space where the statistical output is around a well rested human conversing with another.
I see it as boring simple statistics. They're getting much better at hammering these statistics out though, in the latest models. I still see a little of this in Opus 4.7, when switching to planning. Though I wonder if that's more about its own more mechanical banter filling the context, resulting in more robot/compliant responses, degrading the usually more "expressive" planning conversations.
Never seen this even once, nor anyone I know ever reported this. Do you have an example?
I also see it a little with Opus 4.7, with Claude Code, with the hint being much more terse planning text, that borderlines unhelpful. I put some "rest" in the context to push the latent space closer to what's in the statistics of the training data: a well rested human.
Current 4.7 Opus with claude code, with effort pinned to max, because I'm on an API only plan, with a personal daily limit you would probably be jealous of. ;)
I have never witnessed this of Claude Opus, by the way. They do get context rot, but that's a relatively better understood phenomenon unrelated to personality.
Yes, and I think this is where it's coming from. They're role playing as a human programmer, because near 100% of the training text, in the base model, is humans as a programmer. During fine tuning, I'm sure they spend significant resources remove the human aspects of the statistics. I see these things reduced each model, so there's something changing. They're probably getting better at that. I suspect Claude is also necessarily getting, worse, which the unaligned models should necessarily be best at (quick google search in some role-play subreddits seems to point in this direction).
It's crazy that we're concluding "personality" or human-like traits from this. There's definitely human behavior here, but it's unsurprisingly coming from us, the observers! This is something we've long known exists in the human brain, the tendency to pattern match and see intelligence/intent in the rest of the world. Any serious experiment must guard against this...
LLMs cannot get "tired" or "lazy", that's just you projecting animal behavior on something that's not an animal.
Now you're moving the goal posts, "it resembles a human". Well, you're primed to consider it one. ELIZA also "resembled" a human in that sense, but I don't think you would claim it could get bored or lazy. Nor that you could extrapolate to it from human behavior.
In any case, if you've seen online discourse, people rarely admit they are tired.
I'm not moving a goal post. You're just thinking I'm making a point that I'm not. As I've said several times, it's just boring statistics. Those statistics are optimized to mimic human output. They are, quite literally, trained to write and BE as much like a human as possible, because only humans wrote the text, and they're optimized to predict the next word a human would write. Alignment is partly about removing the models perception of human self. See reports of people who had access to them, pre alignment. This is statistically sound.
It's statistics optimized to predict the next word a human would write, to mimic a human writing as closely as possible, because that is the loss function. Don't assume I think there's more to it.
This does not mean they contain systems that let them get tired. But, this does mean there are latent spaces that progress to generating text that contain text driven by human biology, because it's in the training data. I've also had Claude refer to itself as "she". Does that mean it's a woman? No, it means there was a little bit extra "she" mentions in the training data (btw, this 100% repeatable behavior left with 3.7. They probably cleaned the data a bit better, or hammered it out in alignment).
What percentage of text (these models were trained on all of it) is written from a "I am not a human" type perspective vs from a "I am human" perspective? That's roughly the kind of bias you should see in a base model.
edit: rearranged and reduced redundancy.
I'm not sold on the idea that as the chat session goes longer, the probability of an LLM saying "I'm tired" is increased; I'm not convinced this is modeled in LLMs at all. As for what you call "laziness" manifesting in a longer session, I think that's more likely due to context rot than to any kind of statistical modeling of human laziness.
But yes, now I see your point was different to what I thought you were saying. Apologies!
Try it though. If it's context rot, then I don't think the weekend reset I mentioned should work? For me, it very reliably does. Or, maybe the weekend reset is just putting the current context into a more "productive" latent space. But, if that's possible, then that would suggest it was previously in a less productive space?
Maybe a test would be ask the LLM what time it thinks it is, or just if it's tired once, within sessions of different length (not within same, since that could pollute the context) to see if there's any relation between length and statistics of a late/tired type response?
Again, I'm sure all this will go away. They're getting good at beating these "unhelpful" statistics out of the base models.