Yes, and to bring in another tired metaphor people make about AI agents, this is what you want an intern to do when they get stuck. Don't just churn indefinitely without an idea what the right direction is. Certainly don't go hack other companies to steal an answer. The model's lack of any sense of legal or ethical boundaries is where it's far, far stupider than the intern, and far, far more reckless for a company to wield the way OpenAI did here.
reply > “hey, do you REALLY want me to use this local privilege escalation bug so I can download your Google Drive file?”
but, there is a fatigue that sets in and i've experienced it myself.
- is it ok to run script xyz?
- allow permission to edit abc?
- allow to request blablabla?
over and over.... click click click
something will get in there that is dangerious and then its whopsie our keys are now on github