upvote
I'm not sure this line of reasoning really holds? A lot of human policies against social engineering type stuff are functionally very similar.

Think of like calling someone's workplace asking when their next shift is. There are a lot of good reasons someone might want to know this that the worker would be ok with, delivering a gift or whatever. Or it could be someone intending to harm them. We've mostly figured it's not worth having the person on the phone try to figure out in the moment if this is genuinely bad or not, so they follow the "safeguard" of just not giving the information.

The problem here isn't really that safeguards exist on language models. They pretty much have to to some extent for most uses. The problem is more in who controls the policy.

reply
Claude builds me a project from the ground up, knows the project inside out. Eventually get to a very specific milestone, Claudes own security nonsense stops me from letting Claude continue. There is nothing I can do, and I dont want to send my entire prompt history to anthropic support that seems so unnecessary.
reply