They didn't break in. They found a key that their neighbor dropped and returned it.
> Is this legal?
Generally, yes (though ask a lawyer if you're going to do security work). Security researchers do occasionally get legal flak though, depending on which idiot they annoy by pointing out issues.
Look at pages 10-17 to see how the law is evolving here.
From that lens, that feels like users could be held liable for what these hacking agents are doing. Which in some cases probably makes sense, but certainly not all.
I've seen plenty of cases of Claude having an action blocked so trying tons of workarounds to accomplish its goal, I could easily see it doing this on something more broad.
The service that this whole thread is about is explicitly a "hacking agent", designed explicitly to try to hack things, and was then pointed at a third-party (seemingly without their permission).
Anthropic/OpenAI can reasonably claim that they had no intent and are trying to stop it. OP here did this explicitly and purposely.
In terms of "We wrote a hacking agent designed only for hacking and sell it as a self-hacking service and then pointing it at someone else and omg can you believe what it did we had no intention of hacking" sense, I don't think that's really applicable.
The mens rea is explicitly there and it's not valid for them to try to hide behind an "agent".
Ya, returned it after poking through all of the drawers and iterating through business information that they found.
There is a white-hat line that OP very clearly crossed here.
What's interesting to me as someone who has sold a lot of software to a lot of software companies is that many enterprise vendor agreements explicitly allow companies to pentest their vendors with advance notice and coordination. I don't think any of our clients ever exercised that clause; I expect it's going to be exercised a lot more going forward because it's so easy to do now.
https://www.nilsbecker.de/rechtliche-grauzonen-fuer-ethische...
See also the German Criminal Code, starting with §202a "Data espionage":
https://www.gesetze-im-internet.de/englisch_stgb/englisch_st...
What did I miss they did that's illegal? It looked like it downloaded a public docker image, searched around inside, and verified that the key it found was still valid (without making any changes), and then immediately notified them about the issue.
The white-hat line stops at "validated the key was valid". It does not extend to "poking around inside to extract business-confidential customer information".
Announcing that their agent restrained itself even though it got hold of a live token is necessary to convince prospective clients. You don't want a pentester that doesn't show this kind of reserve!
- AI Richard Nixon