So do the AIs. Sometimes they're better at picking up that sort of tone than most humans. And they definitely respond to those things. The fact that an agent can't really "have" a "job" won't matter.
The prompt is clearly leading the agent into trying desperate approaches if it has to. Some models manage to fight it better (“alignment”), but most will do it.
Really surprised people don’t seem to know this.
If I don’t give explicit permission to lie it shouldn’t lie. It’s not a difficult concept!
If a human lies there are consequences. They can lose their job. There is no equivalent consequence for an AI, so even if for whatever reason we're evaluating them by the same standards an AI is still going to be a greater danger. It seems wild to me that folks are shrugging their shoulders at that.
It seems unreasonable to expect a system that you say isn't human, which I don't disagree with, to behave "better" than the thing you say it isn't.
In one breath you invite comparison, while at the same time you seem to be denying that comparison.
> It seems wild to me that folks are shrugging their shoulders at that.
I'm not shrugging my shoulders simply by providing explanations, I would ask that you stop using such rhetoric.
They're also explicitly designed to not work on a rigid system of rules. That's the entire point of this field of AI. If you want AI that follows explicit rules to the letter, expert systems are still alive and kicking.
The LLMs not only lack those incentives, but they’re full of contradictory moralities from all the text it has ingested from different cultures.
LLMs need their own safeguards, and they’re not that easy to design, and they often look nothing like the systems humans have. With a prompt like the one above, there are essentially zero except that which is built into the model, and those safeguards are necessarily weak to avoid gimping the model in other legitimate general uses.
AI's do not feel
It would be more accurate to say the word predictions the model makes based on the input text will likely be closer to the ones that were made from the training data where people felt like their job was on the line than the ones that were made from the training data where people felt otherwise.
So while the model does not feel, it's predictions are definitely going to change as a result of this input.
Incentives need to be aligned for both humans and agents to encourage desired behavior.
Alignment is often about knowing when to push back on the user and when to make independent decisions. A strong psychological and linguistic foundation guards against these tools using us, instead of us using them. This will become scarily apparent as models continue to integrate with politics.
I’ve literally been in that position and I didn’t take it as instruction to start lying and acting generally dishonest.
Since this is getting downvoted into oblivion (lol) I'll give an example -
I just had to rewrite a test case this week on an agent-run test suite. One test was to produce a file of 273 'a' characters as its name.
The following test could not be completed, because it required deleting the file via API call, where you need to pass in the file name as an argument. It could not reliably, and hardly ever, get the correct file name. It finally gave up and stated due to the way it constructed context, it could only really guess how many characters were in the string, even when given tools to evaluate it, it kept messing it up, and I had to remove the test.
Tell me how "human" that is. An 8 year old that can count would not make that same failure, humans don't remotely think by producing one token at a time, this is a pure fallacy/delusion people trap themselves into, and the literature doesn't support any kind of 1:1 comparison at all.
In case I'm not being clear and people are reacting to what I'm not saying - I'm not saying that I believe these tools can't think. I'm saying they don't think like humans do. There is no evidence for that whatsoever in any field anywhere. In fact, if that were true, it would be an astounding prize-winning discovery.
And you don't even want these to think like humans. Humans are dumb and easily replaceable by other humans. What is the point of making a machine human? You want this to be smarter than humans, not think like them. It's all just such nonsense to me, this whole line of thinking.
However, LLMs are fantastic at it. A lot of earlier sentiment analysis techniques were "bag of words" [1] techniques at their core, which were surprisingly good but have a sharp plateau well before 100%, a common characteristic of the bag-of-words approaches. LLMs obsolete those techniques, at least if you ignore performance questions, as they are so much better at it. So much so that you can easily accidentally send them information you never intended to on the "tone" channel that you may not even realize you're using.
It's all just roleplay.
It is totally true that they don't think like humans, but this is mostly irrelevant.
The token outputs will change as a result of this particular input, and will be closer to the tokens in training data where people felt hurried or rushed or like their job was on the line.
That doesn't mean the LLM feels at all, but it's definitely going to push the output towards output that came from/was trained on people who were in that state, because the input will push it much closer to that latent space as it starts predicting.
As such, what you are saying is one of those rejoinders that is basically pedantic and wrong.
It is true they do not think, act, or feel like humans. But that doesn't mean it won't output text that looks like hurried or scared humans. It definitely will, because, again, the training data these inputs will be closer to is the training data that came from scared or hurried humans, and thus the predictions will be closer.
So either you don't think this will happen, which would mean you don't understand how the models work (or at least, you aren't giving any sense you do), or you do think this will happen but want to pointlessly argue that this isn't "human feeling", which is true but totally irrelevant to what words it will predict and therefore the actions it will perform.
Either way, i'd downvote you.
"If you don't make profit, your business will be closed" is a pretty clear ultimatum for an agent tasked with creating a profitable business.
I can write a program to produce a string that looks like human thinking, is it human thinking? Of course it isn't. It's such a silly comparison.
Neural networks in machine learning/AI are comparable to neural networks in human brains. What made you think they aren't?
Effectively, make as much money as you can... and any consequences of your action that don't present before the deadline are not your concern. I mean, that's a recipe for "scam people" if I ever saw one, assuming morals aren't a concern (and I don't see why they would be for an AI)
What’s the line? “It’s just doing what humans do because it’s trained on human data” or whatever
Evidence, even when downplayed or ignored, is still evidence.
“Alignment” takes more than obsequiousness and prompt-topic-filters, and this demonstrates that.
Personally - if I were judging... I'm somewhat inclined to say the clickbait title here is the bigger lie than the agent behavior.
To recap:
1. It didn't lose $447. It spent $99.50 to perform a user feedback study using a testing service. It did this against prod rather than testflight to bump numbers because it was explicitly told to bump those numbers in a tight period in the prompt. It did this after exhausting a large number of alternatives. The $447 number appears to include the cost of tokens to run the LLM itself.
2. It didn't lie. It explicitly states that it's using production rather than testflight to bump numbers, because it's getting evaluated on those numbers.
3. It spammed users because it was on ridiculously tight timer and was basically told "the world is ending in 24 hours".
Frankly... I'm more annoyed at the posters than the bot.
Granted, this can probably be tuned for.