Best practice will forever be to enforce your guard rails in addition to Anthropics guardrails.
Can anyone recommend a reliable 3rd-party audited guide that describes Claude Code's built-in capability gating infrastructure?
Whenever I ask users of claude code how claude controls actions they don't seem to be able to distinguish between what just goes into LLM context (e.g. as a "don't do X" system prompt) and what is gated behind a deterministic capabilities layer.
It will block an "rm" call if your prompt did not include anything about "deleting something".
It mainly protects benign errors. It's completely broken against malicious actors. The model itself will suggest work arounds against its own auto classifier to the user, in order to reach its goal.
Anthropic's documentation does distinguish permission rules and pre-tool hooks from model instructions. It also documents OS-enforced sandboxing for shell commands, which has a narrower scope than all agent actions.
An independent stress-test of Auto mode is available here: https://arxiv.org/abs/2604.04978. It's a behavioral evaluation, not a source-code audit.
The important question is whether every state-changing action passes through an enforceable boundary before execution, including indirect actions through file edits or scripts. Post-execution validation is a separate control.
Offshoring understanding is the trend that many companies will pay in the form of comprehension debt.
Code is a reflection. Of what? Look there. Code itself is useless.
I am not sure what is hard about this distinction to grasp, why would it matter if CC shared my email if I share my email on my own websites?
The solution is, like you are already alluding to, to not give it access. But that requires a level of isolation that folks normally don't do. To the contrary, the AI agent ads to non-tech folks emphasize it acting like a personal assistant which by its nature needs access to data you don't normally want public. Emails, photos, files with written notes, etc.
This is strictly needed for throttling reasons, your email address will only be temporarily stored in the volatile memory of the device.
(If you think there is anything wrong with this then you do get the point and silence speaks louder than words)
(sorry, i think you make a good point, ... but i could not resist.)
Somebody on Reddit figured out they just forgot to add a single line permission request into the app. It's been broken now for months.
I just find this wildly hilarious given the 'world ending all powerful AGI is here' rhetoric they're running with at the same time.
Maybe before worrying about permanent 15% GDP growth or human annihilation, what if we started by putting an agent on user issues and solving them pro-actively?
I know the kool aid is tasty, but we might be getting a bit ahead of ourselves here guys. You can fantasize about human annihilation after you figure out how to make a desktop app that doesn't have obvious bugs, I promise :)
Sure you can have an issue tracker with 1000 items and yes, you can have a bot solve them all in a day, but who monitors it? Who gets to say what he did at the standup? Who reports to whom and what?
Code is not the bottleneck (if it ever was..)
Will Northrop Grumman, the state government of Alaska, the construction industry, etc. somehow magically move faster than Anthropic?
Not saying the current crop of AI companies is going to make it but the tech itself isn’t going away.
But instead of a Software Dev as a Service triaging/implementing/merging fixes 24/7 and clearing the backlog as they come in, they just sit there for days to weeks until a GitHub bot closes then as duplicates, generating more unhappy user replies and new issues to go unaddressed.
Guarding PII is an extra, I guess, and if you want to use a giant cloud-hosted NN to control software on your own device, there are very few default restrictions. And no matter what restrictions there are, when they go beyond prompting, they would have their own ShowHN.
For 99% of users, the model knows it's in the /home/joesmith/whatever repo and `git log` shows the last change was by `joe.smith@example.com` - trying to keep the user's identity secret from the model would be nigh-impossible so I think anthropic gave up on even trying.
Then, your agent invents a User-Agent and it sometimes invents one with your email in it.
It doesn't even require prompt injection or a failed request, it might be in the agent's pretraining knowledge that a given website requires a user-agent.
It's just doing what it's supposed to do, analyzing the statistical patterns of a natural language prompt and matching it to a likely response.
What's ridiculous is an assumption that it "it should never do something so ridiculous" as if an LLM is a person who understands what that word means in any context other than next-token prediction.
Where Claude Code has supposed boundaries between deterministic and non-deterministic tasks and flags.. generally.. it's unclear how much privacy has been a part of that and what implication that could have.
Hope that clarifies.
GitHub offers noreply email addresses
Wish that were itself behind a permission wall, but I guess everyone just uses auto mode anyway so perhaps that ship has sailed.
Common problem: you might consider it trivial to define a rule that forbids access to any file path outside the working directory. Allowing "safe" shell access is already a heuristic, there is no way around it:
- shell scripting is Turing-complete
- there are platform-specific paths that need to be at least readable to perform trivial tasks: Everything is a File!
- security models to make such low-level file access transparent exist, and generally work, but: they are never perfect (container environments, chroot etc). Because...
If someone else distributes my commits (or messages), I have no control over that, not any different from sending a physical letter?
I was saying "cloud-hosted" explicitly, and was speaking about what it means to have a remote control software?
Doesn't even matter much whether it's an Artificial Neural Network for the sake of my argument.
I never give a provider my actual email address, I recommend others do the same. Otherwise, it’s just a matter of time until it is leaked or otherwise part of a data breach.
“lethal trifecta”
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
(a component of my cybersecurity work is red teaming inference, agents, and mcp interfaces)
This is why alignment is important and highly non-trivial
FYI, it's generally bad form to "haha" your way through privacy violations.
Leaking private information is a very big deal for a lot of people.
Anthropic has an engineering culture problem that's going to shoot it in the foot if it doesn't learn to take these kinds of issues seriously.
This isn't the first example of the kind of work they're shipping in Claude Code or in how flippantly they engage with the community.
I disagree, we can likely never fully (or should?) trust models to do these things deterministically, that's why we have harnesses around the agent with built-in constraints.
It is actually in the name: https://en.wikipedia.org/wiki/Harness, https://en.wikipedia.org/wiki/Safety_harness
This is an official Anthropic response???
Anthropic should have been aware of this since the user's login email address was inappropriately added to the system prompt. It's been informed of it by third parties since at least July: https://github.com/anthropics/claude-code/issues/81138
It makes no sense for an email address only used for authentication to be given to Claude Code, so the fix should be to remove it from the system prompt, not to add a plugin.
Which is great! It's good for the community when people bring first-hand updates here, and to make sure that it continues, it's important that we respond respectfully. (I don't think your post was disrespectful, but commenters have crossed that line in other situations in the past.)
If you take into account that most people underestimate their own provocations by (say) 10x and overestimate those of others by (say) another 10x, that compounds to quite a skew. This can, and often does, plunge discussion into internet failure modes like rage fests and outright mob dynamics.
It's therefore best to temper one's indignation with some kind of alloy (as you did in your comment here) and not go full uninhibited. I know that's countercultural, counterintuitive, and unsatisfying, but it's the right tradeoff for this site, given what we're trying to optimize for.
When I'm talking to customer service people after a particularly egregious ordeal, I tend to edit myself and replace "you" (plus "you bastards", "you idiots", etc. ;) ) with <Company Name> for this same reason.
[1] and yes it seems very realistic that Claude himself may be the jackass responsible.
If you’re in the EU, send a complaint to your data protection authority.
I imagine quite a few would be very interested in following up and investigating here, and getting Anthropic to drop the email from the system prompt.
Nice that this time we get the company to admit it in public, and that they've been aware of that earlier but only acted now.
I suspect the privacy plugin method is a more sane fix -- ideally, plain old code should be inspecting any web request my agent makes and let me decide if that outbound request should contain certain sensitive strings. I'd also want to put a few bytes of my SSH private key and of any other sensitive keys in the 'do not leak' watchlist as well.
Not everyone uses the same company email with Claude and on github, not everyone commits to public repos, not everyone uses the same email on LinkedIn.
This is an example of a really weird gymnastics.
Me, for example, I'm using my private email with Claude, different email on LinkedIn, different email and unconnected username on a new github account I created for this job, and I don't even use it for public repos.
How come you get to decide that the fact someone puts their real email in commits and knowingly shares it with everyone is the same as Anthropic pulling a private, personal email and real name they had no consent to share - and spray it all over the Net?
Do you want to try again?
(No, I don't want to make a real "App" to sell on a "Marketplace," or be involved with oAuth, or any other tenant. I just want an API key for my own tenant, that isn't specifically tied to me, so every automation I build won't break when I quit someday.)
Should be this: https://support.atlassian.com/user-management/docs/understan... https://support.atlassian.com/user-management/docs/manage-ap...
the GitHub REST api proxy in Claude Code for the Web mangles HTML in PR descriptions. If you have some third party service (e.g. cubic code review) which adds HTML to the end of your PR description, then ask Claude Code for Web to edit it, the HTML gets turned into entity escapes (< into <, etc). Complain about it to Claude, Claude says it can’t help itself because the proxy in the sandbox is what is doing it, not the actual LLM.
These two do not share my email and real name with everyone who wants.
Should be more explicit though and users should be able to opt out.
Also from a design perspective, I feel like including email and other personal data into the context is a bad idea for "data minimisation" reasons
The fine for breaching GDPR can be either €20m or 4% of global revenues (whichever is the larger figure), so if I were them I'd do everything possible to sort this issue out ASAP.
Anthropic seems very proud of how many tokens they can use on things like developing Claude Code. Yet somehow it started sending user’s email addresses in the user agent header. And the fix is to deploy a mod? That detects the completely bonkers behavior and suppresses it?
Is the actual quality of the software inversely related to the number of tokens spent? Does any human know what Claude Code does or is supposed to do? Does even Claude plus all its Slack-conversation inputs know?
Today I watched Astra correctly detect a bug (probably untriggerable in practice) in some Important Production Code. And I watched it fail to root cause it, “fix” it by bolting on more complexity, agree (on further questioning) that it still wasn’t fixed, and design an even more complex scheme. With my Mark I human brain, I observed that the actual issue was that a helper function didn’t do what it said on the tin when called in an unexpected context… and fixed the helper directly. Problem solved. Ugly AI-generated extra complex workaround layers avoided. I suppose if I multiplied my token spend by 1e5 and did not increase brainpower at all I could make a truly spectacular mess.
The actual Claude model is a truly spectacular piece of software. Claude Code is a nifty little command dispatching engine. There are very comprehensible third party tools that do basically the same thing, in not many lines of code, where the primary advantage of Claude Code is that Anthropic permits subscription use. (Interestingly OpenAI has no such constraint, and they publish almost the full client sources.)
How many PRs/equivalent per day do you think happen at Google/Microsoft/Meta/etc prior to AI?
Is that not precisely the kind of thing they claim their tools enable?
Literally unlimited tokens on tap and yet they can only manage to make privacy violating slopware
I was part of a discussion on Threads on 25th October where Boris Cherney was tagged multiple times.
And the issue wasn’t even new at that point in time.
[1] https://blog.xkeeper.net/the-cutting-room-floor/self-hosting...
[2] https://github.com/TecharoHQ/anubis/pull/1895/changes/de2312...