Not saying you’re wrong, but I think the bigger issue is how easy it seems to be for models to hack companies, even ones with generally ok security. Most tech companies are not doing continuous, deep security audits of their code and infrastructure. Dependencies are not updated quickly as RCEs are discovered. (And any org with a slow release process where it’s hard to be confident that an OS or package update won’t break something… is in even more trouble.)
replyThe only reason more companies aren’t exploited is because human attackers don’t have the time and energy to waste on trying every play in the book, or attacking lower value targets.
They key lesson I've picked up from the past ~4 months is that models are now good enough that, if there's a security hole, they'll brute force their way into finding it.
replyThe only solution that makes sense to me is for defenders to get to point these models at their own code to find the holes before the attackers do.
But that's hard, because how do you limit access to defenders and restrict access to attackers? Attackers aren't exactly honest people.
These attacks are also incredibly loud. Many attackers are motivated to operate very quietly. We haven't seen any tradecraft from these machines, it's all noisy and bombastic.
replyWhen we see them mount a quiet backdooring campaign, like the XZ-SSH attack, or something like Stuxnet, then we'll have real problems.