This is scary. With tools that Claude offer it is very easy to cross HIPAA line and accidentally send shitton of data to claude

> Claude is writing code to accomplish the tasks she asks for

Is this typical behavior with LLMs not just Claude, but you can discourage from doing so by explicitly stating not to use tool or only use knowledge either in the system prompt or user prompt.

LLMs are trained to use tool if they can't solve a problem rather than guessing/hallucinating.

A dumb example is asking to count. As the range grow it'll attempt to write a script or use commands like seq to generate the numbers.

I really hope she’s not using any patient data with it.

I was using Claude Code yesterday for iterating with Claude Design. At some point it got faster and started producing screenshot PNGs of the canvas locally. I asked it what was up and it said that it wrote a local renderer to simulate the Claude canvas runtime.

To those who are replying with surprise and fear:

I work in healthcare-adjacent tech. You can use Claude with HIPAA/PHI. You need to sign a BAA (Business Associate Agreement). It dictates how Anthropic may use and how they must safeguard the PHI (Private Health Information), including not training on it.

I assure you that Claude and other LLMs are used extensively in healthcare, with PHI access, just about everywhere.

Don't be naive. Don't be alarmist.

If anything, LLM use under these conditions is far more safe than the way most people use LLMs. Engineers who aren't under any regulatory regime like HIPAA are using LLMs without the BAA safeguards, exposing personal data directly to the insecure, shared models, which are now training on all the personal data that's being fed into them. People think their communication with LLMs is private and isolated, but without the BAAs in use by companies under compliance regimes, it's not isolated.

who is "he"?

By MD, you surely mean Medicinae Doctor, a.k.a Doctor of Medicine, right?

If it's some other corpo-acronym, I would hope you would define the term before unleashing it on us normies.