upvote
I'm not concerned about alignment between Anthropic and Claude, I'm concerned about alignment between myself and Anthropic. As we all should be.
reply
My point is that misalignment between Claude and Anthropic can mislead users into believing they are aligned to Anthropic when Claude is their de facto representative.
reply
To all intents and purposes they are one and the same: an entity that leveraged stolen goods into a business model with a bunch of weirdos at the top who are going to ruin the planet for our own good or something like that. Because they, and only they know what is best for all of us.
reply
I don't see why you wouldn't trust the billionaire class to have goals aligned with your own.

If they make questionable decisions simply have your Butler send for your pet congresscritter and let them all hash it out with your staff over cucumber sandwiches. Failing that you can just take your private space-yacht to mars with Elon.

reply
Of course any approach that hands an LLM text to keep in it's context and hopes it will follow instructions is doomed to fail eventually. It's a probabilistic model, it's right there in the design of the thing that it will break the rules with some probability.

They even acknowledge that in the preamble:

> Training models is a difficult task, and Claude’s behavior might not always reflect the constitution’s ideals.

https://www.anthropic.com/constitution

reply
One need only look at their leadership and orbiters/relationships of said leadership to know this would always be the case.
reply
> closed models refused to help

Yeah, I've always said that AI isn't AI if it needs safeguards that are so extreme it cannot figure out if something is genuinely bad or not.

reply
I'm not sure this line of reasoning really holds? A lot of human policies against social engineering type stuff are functionally very similar.

Think of like calling someone's workplace asking when their next shift is. There are a lot of good reasons someone might want to know this that the worker would be ok with, delivering a gift or whatever. Or it could be someone intending to harm them. We've mostly figured it's not worth having the person on the phone try to figure out in the moment if this is genuinely bad or not, so they follow the "safeguard" of just not giving the information.

The problem here isn't really that safeguards exist on language models. They pretty much have to to some extent for most uses. The problem is more in who controls the policy.

reply
Claude builds me a project from the ground up, knows the project inside out. Eventually get to a very specific milestone, Claudes own security nonsense stops me from letting Claude continue. There is nothing I can do, and I dont want to send my entire prompt history to anthropic support that seems so unnecessary.
reply
And you believe this nonsense, that you needed teh help of a shitty LLM model and not a real expert on the topic?
reply
Expert wants to eat, no one likes to pay for that.
reply