undefined

[-]

Thanks for the clarification, in that case the text is indeed really weak. Does that system work in practice, or are companies just claiming they are HIPAA compliant with close to no actual auditing mechanism?

by giaour3 hours ago|

[-]

It's been a few years since I worked in this space, but HIPAA doesn't really work under the same kind of legal framework. Oversimplifying here, but basically HIPAA defines what constitutes personal health information, how such information may be used, and establishes monetary penalties for improper use and unauthorized disclosure. The law doesn't have any certification standard, no more than the prohibition on stealing cars does.

Maybe there's some kind of third party certification system to support signing information sharing agreements ("BAAs") with other health information systems. I worked at CMS on first-party stuff so I'm not really familiar with how it works in the private sector.

by tptacek8 hours ago|

[-]

You get that the technical controls in SOC2 are also extremely weak, right?

[-]

Sure, yes. The way I understand SOC2 relies on the auditors to set the effective standard. So it really depends who audited you

by tptacek7 hours ago|

[-]

SOC2 auditors are accountants. A SOC2 auditor verifies only that you're doing what you say what you're doing.

by kevin_nisbet6 hours ago|

[-]

And the way they verify you are doing what you say you are doing is by asking you to provide evidence, which is usually pretty easy to demonstrate that a policy was followed once or twice, a lot harder for them to pick up consistency issues or exceptions.

by randerson3 hours ago|

[-]

I've had SOC2 auditors choose a random commit from our GitHub history, then ask to see the associated Jira ticket, logs from the build and deployment, etc. Hard to reliably pass an audit if you don't know which changes they'll drill down into.

They also asked for proof of system-enforced processes (e.g. GitHub branch protection rules and the setting for enforcing peer review for each change) which were basically proof of consistency.

by akerl_32 minutes ago|

[-]

An example of what the parent commenter meant is more like:

1. You write a DRL that says “we do a disaster recovery test annually to ensure that we can restore a production backup”.

2. It takes you 20 tries in 2026 to do a successful restore because your find out the first 19 times that your backup is broken and incomplete.

3. You never have to mention the first 19 tries to the auditors when you prove you did do a successful DR restore.

by tptacek2 hours ago|

[-]

They do that because in the DRL process you specified a change management process involving Github and Jira. If you specified a different process (for instance, Post-It notes applied to the bathroom wall), they would randomly ask for evidence about those Post-It notes.

That's what we're talking about when we say virtually any tool you can come up with will satisfy "vulnerability scans". For Cloudflare, it was nmap. I think they're right about this.

[-]

Obviously, yes

by akerl_7 hours ago|

[-]

A SOC auditor who tells you that you can’t use an nmap scan to meet SOC2 obligations is a bad SOC auditor, because they’re attempting to enforce a constraint on you that SOC2 does not.

But the far more likely thing is that a medium SOC auditor, upon being told “we do our vulnerability scanning with nmap”, would say “I haven’t heard of nmap. You should use Tenable,” and if you’re letting SOC auditor drive your engineering you’d make a mistake and accidentally think that meant you needed to change your answer for SOC2 and go buy Tenable licenses.

[-]

The whole thread drifted way too far from a very mild push back I had regarding the claim « any automated process that can plausibly be described as instrumental in finding some kind of vulnerability is a "vulnerability scan" ».

My experience is that no, SOC2 auditors won’t consider literally any automated process of that sort as compliant. Which in no way implies the auditors are forcing you to use a licensed tool or driving your engineering.

I will stop that thread here, I don’t think that exchange is productive

by _hyn32 hours ago|

[-]

If SOC2 relies on competent auditors (and you're right, it does), than it is an ineffective standard (and it mostly is).

by tptacek8 hours ago|

[-]

No? Like, wildly no? This is a big part of why you pay for the most respected auditors.

by dgellow8 hours ago|

[-]

I guess we had different experiences. The ones I interacted with were ok and wouldn’t have accepted a simple nmap here

by tptacek8 hours ago|

[-]

I'm not being snarky when I say that not getting your automated vulnerability scan, whatever it might have been, past your SOC2 auditors is a skills issue. SOC2 audits are not technical and the vulnerability scan control in SOC2 is categorically not meaningful. Cloudflare wrote a whole post about this.

[-]

FWIW I agree that SOC2 for automated vulnerability scans has a really low bar and isn’t too meaningful. At no point did I defend SOC2 here. The bar I’ve seen is above “just an nmap”, which is pretty bad standard IMHO. You seem to be reading way too much in my comments

by tptacek7 hours ago|

[-]

I brought up nmap. You said you'd expect respected SOC2 auditors to reject it. I don't just think that's not true, I know it not to be true.

[-]

I know, that’s already established. I already acknowledged we had different experiences. I have no idea what you’re pushing for at that point

by tptacek6 hours ago|

[-]

Just to clarify, this is a bugbear of mine. It's nothing personal with you, but I've spent the last 6 years or so evangelizing the idea that people should minimize their SOC2s and not get pushed around by auditors or evidence collection platforms like Vanta, because that drives a lot of terrible security engineering, and the hypercompetent best-staffed security orgs in the industry all push their SOC2 auditors around.

Compliance and security are entirely different practices in a well-run firm. Security can inform compliance. Compliance should not inform security engineering.

If you search my name and "SOC2" in the search bar below, I've expanded on this quite a bit.

by rmccue5 hours ago|

[-]

As just one data point here, let me say thank you for all your writing on it; it was super useful to have things to point at to say “we don’t have to just blindly do a thing the auditor suggested!” for our SOC2.

by john_strinlai6 hours ago|

[-]

tptacek just hates soc. its probably not personal.

by tptacek6 hours ago|

[-]

We got some value from it! I just think it's important to remember what it actually is, rather than axiomatically deriving what you think it should be.

by 7 hours ago|