upvote
For it to fix the bug it has to identify the bug. If the bug is a security vulnerability then it will have to identify the security vulnerability to fix it. What's the alternative, have it ignore vulnerabilities/bugs? It wouldn't be a very good coding companion in that case.

I'd pay less attention to the prompt and more attention to the output when interpreting this story. (I'm not saying I agree with the decision, but this is how they are looking at it.)

reply
It basically as if you asked it to find ways to enter someone's house and it refused.

But then give it exact copy of their house, ask to secure it, which it does and look at what it secured to find out how to get into the original house.

reply
yeah, they don't want it to be able to find security bugs that can be exploited.
reply
No. Anthropic spent months telling the world that LLMs are nukes and then got surprised when they got regulated like nukes. They specifically argued that Mythos was too dangerous to release publicly because it can find security bugs, and then released a watered-down version (Fable) that was supposed to recognize when it was being asked to find security bugs and downgrade itself to Opus. Then Amazon figured out that it'll happily find security bugs as long as you don't mention you're hunting security bugs. So the US government put an export control ban on Fable, because that's what Anthropic begged them to do.

To add to this, Pete Hegseth wants to make an example out of Anthropic because they refused to amend their contractual language to allow the Department of Defense[0] to make fully autonomous kill drones. This is, of course, a really petty and stupid dispute, but the hallmark of the Trump Administration is engaging in really petty and stupid disputes with the full faith and credit of the United States backing them. This is exactly the kind of administration you do NOT want to give rhetorical ammunition to, and Anthropic handed them a whole ammo belt.

[0] It is always ethical to deadname governments. Especially when they aren't even legally allowed to change their own name.

reply