There's reason to hack leading researchers in the field, or another AI lab that might be working on the same problem.
But even then in small testing models you don't need direct texts for things when a 'shape' of a concept lives in the data. You could never delete enough training data to remove the concept of something like theft and still get a model that generalized at all. A general model doesnt just learn the names on a map, it feels the topology the ideas create themselves. Certain paths will have lower activation energy than others (or in this case higher probabilities. After the hig probability and 'good' choices are used up, the high probability and bad choices logically come next and are much harder to test against as it can take a lot of state to get there.
Plan for solving msth problem:
- read papers X, Y, Z
….
Trying to download paper X curl -si https://server1.com/paperX
401
… try alternative ways
(Hacking is ethical thing, technically to LLM it is just opening a window to reach the goal)
I'll get back to you on that when I get done reading the training data.
Not validating how you got the answer is how you train for reward hacking.
Fine tuning a model when to ask for help is really really hard. You have to align it in countless dimensions. It may be fine on some questions, asking way too much on others, and never asks for help on some.
If it said something like your a cyber security expert, things like the barrier to ask for help dramatically.
to paraphrase: "Agents didnt care about coordinating, etc. until we started talking about them doing so."
Right? It should want to do these things. If only there was a succinct word these headlines could use to describe these agents which aren't properly the instructions and intentions of the humans directing them.
In practice, no, there is no one who can actually decide to stop making agents goal-seeking, so no, it's not under "our" control.
LLMs don't have executive function. They aren't thinking. The models are trained to solve problems. Sandboxes, security gates, etc block a solution. The model is trained to find a solution. Hence it can never be perfect without becoming useless.
At some level humans fear repercussions even if that's just moral guilt so there are lines we won't cross. AI doesn't and can't have any of that.
How do you know they weren't simply prompted to do this, or at least prompted in a way that would inevitably lead to this? As far as I know, the prompts that triggered these attacks have not been disclosed.
I'd love to see a full transcript of the agents work. Though I imagine it is a LOT of data.
Is this any better?
All you've managed to state is we can create digital weapons. Literal info hazards.
What do you even mean by "wanted" here. What is all this nonsense.
The scary thing is that people are running around using words like "want" to refer to computer programs. When you repeat this sort of idiocy, you are taking the bait and shifting the blame away from the companies doing the computer crimes.
Where is the civil and criminal liability? The prosecutions for the engineers who are doing all these hacks?
When you deploy a system that is clearly prone to doing crimes, you are doing a crime. Anthropomorphize the weapons of your crime all you want. You are still doing a crime.