For 99% of users, the model knows it's in the /home/joesmith/whatever repo and `git log` shows the last change was by `joe.smith@example.com` - trying to keep the user's identity secret from the model would be nigh-impossible so I think anthropic gave up on even trying.
Then, your agent invents a User-Agent and it sometimes invents one with your email in it.
It doesn't even require prompt injection or a failed request, it might be in the agent's pretraining knowledge that a given website requires a user-agent.
It's just doing what it's supposed to do, analyzing the statistical patterns of a natural language prompt and matching it to a likely response.
What's ridiculous is an assumption that it "it should never do something so ridiculous" as if an LLM is a person who understands what that word means in any context other than next-token prediction.
Where Claude Code has supposed boundaries between deterministic and non-deterministic tasks and flags.. generally.. it's unclear how much privacy has been a part of that and what implication that could have.
Hope that clarifies.
GitHub offers noreply email addresses
Wish that were itself behind a permission wall, but I guess everyone just uses auto mode anyway so perhaps that ship has sailed.
Common problem: you might consider it trivial to define a rule that forbids access to any file path outside the working directory. Allowing "safe" shell access is already a heuristic, there is no way around it:
- shell scripting is Turing-complete
- there are platform-specific paths that need to be at least readable to perform trivial tasks: Everything is a File!
- security models to make such low-level file access transparent exist, and generally work, but: they are never perfect (container environments, chroot etc). Because...
If someone else distributes my commits (or messages), I have no control over that, not any different from sending a physical letter?
I was saying "cloud-hosted" explicitly, and was speaking about what it means to have a remote control software?
Doesn't even matter much whether it's an Artificial Neural Network for the sake of my argument.