Is this what kids call rage bait these days?
If MS Word had some kind of systematic bias that made it likely to corrupt the notes taken in it such that they tend to be more bloody battle plans, then that would indeed lead to some responsibility on the MS engineer's part for any mistakes that result from this bug. Similarly for Ford engineers if their vehicles had some bug that made them more likely to lead soldiers riding in them to the wrong destination.
If your system is designed to help choose targets, and it chooses the wrong target, especially with such tragic consequences, then your system is likely not fit for purpose and you should be held responsible for selling it to the army. And that is assuming it didn't actually have some bias + misalignment problem where it intentionally tried to kill children, and selected this school specifically because it gave some plausible deniability. For all we know, it's even possible that some bias like this was known about inside Amthropic and hidden from the public - unless an external investigation is made, that can't be ruled out.
If you provide tools that help an organization kill easier, and possibly lazily rely on your inaccurate tools' judgment, and you do not care about the outcomes of these tools' uses, that's on your soul, too.
Yes, causality and attribution are hard sometimes, but it may have been that without it, hundreds of humans would be alive right now, and he doesn't even know. Maybe he should talk to the kids' parents about what alignment means.
I feel that everyone in this small AI bubble (compared to the "normies") is busy trying to take advantage of AI to surpass everyone else, while fearing being surpassed by everyone else.
Most people assume LLMs are just a tool for making money, an arms race over wealth and social rank. But those like Dario seem to be worried that LLMs could potentially be used as automatic rifles by some dedicated psychopaths. Whether it could be that dangerous, or just an exaggeration, or we should have everyone armed, require licenses to own one, or outright ban them, is debatable. But I can understand his point.
All I can say at the moment is that this is more subtle than the superficial conspiracy/black-and-white narrative.