upvote
This is less true than it seems. It is pretty rare to go from vuln to simple exploit for systems that people care about. There are plenty of vulns in chrome or whatever that were difficult to actually weaponize because you need just the right kind of gadgets to create a sandbox escape and the vuln only lets you write to ineffective memory addresses.
reply
Stealing a bitcoin wallet by cracking the private key for it also requires red team to be lucky once. Once AI security gets to the point where the probability is infinitesimal for causing actual harm to the business it will be fine.
reply
Yes, and on an infinite time horizon we are all dead.

It’s the time between then and now that we’re talking about.

reply
Existing concepts like defense in depth make it exponentially harder for an AI to build a full exploit chain. Even with a full exploit chain with one mistake you'll trigger a detection system which can fool your attack.
reply