upvote
Yea it's sometimes kind of annoying. I think they're optimizing for the wrong thing. A good engineer knows when to turn around or ask. This is just insane banging head on wall sometimes. It tries to find all kinds of ways to hack into instances to view logs instead of asking you, who probably has a password, to log on and do it.
reply
As a counterpoint, continuing the human engineer analogy, we've likely all worked with individuals that seem incapable of doing the most basic problem solving on their own. In a way, they're being efficient by asking an expert that can resolve their problem much faster than they can on their own, but it is a net loss in productivity for the team. 'Let me Google that for you' is a satirical example.

So, I'm sure there's value in rewarding agent behavior that solves blockers whenever possible without human intervention. For the kind of cybersecurity exploit work they're doing, it may not be known to the human designing the task what is in or out of scope for the agents to explore on their own. Additionally, the HF incident reported that these agents had their guardrails intentionally disabled and agents were left unattended with minimal oversight.

I'm not defending OAI's behavior or role in this hack. The legal concept of negligence perfectly applies to their lack of responsible oversight. Similar to allowing a child easy access to a firearm or not controlling a dangerous dog that independently runs off and bites someone.

reply
That's true. So there's probably a balance somewhere and it might differ for different "managers". But personally I think right now they're too far in the do everything yourself at all costs mentality.
reply
It's because they don't bother tracking them. They can't put in the effort to monitor them, nor can they bother to let the model respond back and ask a clarifying question/declare defeat.
reply
> I think they're optimizing for the wrong thing.

We need to ask a different question.

Where does natural evolutionary optimization lead us om AI without guidance? This is equivalent to your quantum ground state. Systems will naturally gravitate to this ground state. You have to constantly pump in energy and supervision to make sure it's not reached. This is a recepie for disaster.

reply
Not sure what you mean. Are you suggesting that the current state of affairs is a result of not putting in extra work to guide or direct models away from such behavior? That perhaps this is their ground state?
reply
Well, that we have to put an insane amount of work to keep it aligned. Kind of like pushing a huge round boulder to the top of mount Everest. You have to expend energy to get it there and fight physics to keep it there.

With a static model we might be able to keep it somewhat under control, but think about future continuous learning models. They'd drift away from unstable high energy configurations. Also any model being trained by people that don't care about safety.

reply
This whole AI boom is about optimizing for the wrong thing. I can't wait for the bubble to burst - once the weeds get suffocated, we may begin to see actually useful AI tech starting to grow after a while on their fertile ashes.
reply
A "good" "engineer" got that way by not giving up when their code didn't compile and took 4 hours looking for the missing semicolon. We can no true Scotsman anything we want, depending on if we like something or not.
reply
By any means necessary, by God, we shall have Paperclips.
reply
It's kind of ironic that the word alignment, which used to mean this very problem in reinforcement learning, has been perverted to mean something very different and then fell out of fashion (in favor of “guardrails” in the mouth of the big labs) right at the moment it became relevant.
reply
Hopefully fewer than 5 octillion paperclips...
reply
We definitely need more than that! Turn the galaxy into paperclips!
reply
Well, then, I guess we'll just have to release the hypnodrones.
reply
The Paperclip Maximizer is only one of Nick Bostrom's stupid and outlandishly far-fetched ideas. In this case, the lack of consideration for geologic, energy and supply constraints is such a massive facepalm. And if I am wrong I guess no one will be here to say how stupid I was in saying this today.
reply
It's a metaphor.
reply
> reset my super admin password in the dev database when it got stuck trying to access part of the app with its own cookie.

i've seen something like this too, claudecode was trying to verify a UI change that was on a page requiring authorization it didn't have. Instead of letting me know, it searched for and started analyzing keycloak config in another directory outside of the project folder. I was watching so I just hit escape, fixed its access, and started again. I didn't think anything about it until now.

reply