Nothing saves if human is completely out of the loop
replyOr just the natural consequence of adopting "move fast and break things" from top to bottom for 20 years, a house-of-cards methodology that was destined to fall in on itself whether or not such models ever arrived.
replyFormal verification doesn't solve all the problems, but alongside concerns like correctness, restrained surface area, supply chain awareness, treating PII as toxic waste to avoid rathee than treasure to collect can leave projects/orgs in a much better place than they let themselves get too.
Long way to go from here though.
This is one of the reasons anthropic nerfs their models. They want you to have an enterprise contract with some spend before they’ll allow you a couple of seats on cyber models, which you need since attackers are much more nimble with ablated open weights models.
replyAs a side note to the "nerfing", I worked with a journalist on the analysis, and she wasn’t comfortable running `curl` on the command line. So she asked the AI to do it -- but it refused, citing concerns about cyber abuse.
replyOn the one hand, that’s kind of encouraging. On the other, it feels like the unskippable “don’t copy this movie” warning on a legally purchased DVD -- the one that gets removed from pirated versions.
It seems to punish the wrong people.
The ultimate capitalist model, create a never before experienced problem and sell the solution on a never ending subscription. Progress!
reply