However, the broader idea of supply chain attacks remains challenging and AI doesn’t really matter in terms of how you should treat it. For example, the xz-utils back door in the build system to attack OpenSSH on many popular distros that patched it to depend on systemd predates AI and that’s just the attack we know about because it was caught. Maybe AI helps with scale of such attacks but I haven’t heard anyone propose any kind of solution that would actually improve reliability and robustness of everything.
[1] Fully Countering Trusting Trust through Diverse Double-Compiling https://arxiv.org/abs/1004.5534
And the thesis linked above seems to go beyond simply "use a trusted compiler to compile the next compiler". It involves deterministic compilation and comparing outputs, for example.
TLDR: what I said is only foolish if you take the absolute dumbest possible interpretation of it.
You're right though. There's been talks of a big global hack attack for a while now.
Nothing is safe anymore. Keep everything private airgapped is the only way forward. But most of our private and personal data is in the cloud, and we have no control over it or the backups that these companies keep.
While LLMs unlock the opportunity to self-host and self-create your infrastructure, it also unleashes the world of pain that is coming our way.
The kernel is not just open source, it's a very fast-moving codebase. That's how we win all wars against AI-authored exploits. While the LLM trains on our internal APIs, we change the APIs — by hand. When the agent finally submits its pull request, it gets lost in unfamiliar header files and falls into a state of complete non-compilability. That is the point. That is our strategy.
In the end we need fully deterministic, 100% verifiable, chains. From the tiny boostrapped beginning, to the final thing.
There are people working on these things. Both, in a way, "top-down" (bootstrapping a tiny compiler from a few hundred bytes) and "bottom-up" (a distro like Debian having 93% of all its packages being fully reproducible).
While most people are happy saying "there's nothing wrong with piping curl to bash", there are others that do understand what trusting trust is.
As a sidenote although not a kernel backdoor, Jia Tan's XZ backdoor in that rube-goldberg systemd "we modify your SSHD because we're systemd and so now SSHD's attack surface is immensely bigger" was a wake-up call.
And, sadly and scarily, that's only for one we know about.
I think we'll see much more of these cascading supply chains attack. I also think that, in the end, more people are going to realize that there are better ways to both design, build and ship software.
The possibilities within a good threat could be catastrophic if we assume so, and if we assume nation-states to be interested in sponsoring hacking attacks (which many nations already do) to attack enemy nations/gain leverage. We are looking at damage within Trillions at that point.
But I would assume that Linux might be safe for now, it might be the most looked at code and its definitely something safe.
LLVM might be a bit more interesting as it might go a little unnoticed but hopefully people who are working at LLVM are well funded/have enough funding to take a look at everything carefully to not have such a slip up.