upvote
I might've been bamboozled, but I recal that happening due to outdated satelite images, not a mistake of the AI. Thus, a human would've likely made the same mistake.
reply
[flagged]
reply
There is a huge difference between your product being used for basic military operations like taking notes or getting from A to B, and your products suggesting military operations that kill hundreds of schoolchildren, an obvious and horrific war crime.
reply
It's a mistake. If it was done by an auto-correct mechanism, would we still say "maybe don't sell your word processor to the army if you can't vouch for its auto-correct system!!!"? Besides, "taking notes" can literally be listing targets, and "getting from A to B" can be the road to killing them.
reply
Again, a product that facilitates the taking of notes, even notes that entail the killing of human beings, is fundamentally different from a product that exists to suggest which human beings to kill (or at least which places to destroy, regardless of whether human beings happen to be there or not).

If MS Word had some kind of systematic bias that made it likely to corrupt the notes taken in it such that they tend to be more bloody battle plans, then that would indeed lead to some responsibility on the MS engineer's part for any mistakes that result from this bug. Similarly for Ford engineers if their vehicles had some bug that made them more likely to lead soldiers riding in them to the wrong destination.

If your system is designed to help choose targets, and it chooses the wrong target, especially with such tragic consequences, then your system is likely not fit for purpose and you should be held responsible for selling it to the army. And that is assuming it didn't actually have some bias + misalignment problem where it intentionally tried to kill children, and selected this school specifically because it gave some plausible deniability. For all we know, it's even possible that some bias like this was known about inside Amthropic and hidden from the public - unless an external investigation is made, that can't be ruled out.

reply
How intelligent and autonomous does a murderbot have to become before you stop absolving its creator and seller from any responsibility?
reply
If the US military purchased camera guided smart munitions with a miscalibrated camera, such that it ignored adult targets and instead specifically targeted children, would that also be a simple mistake per your argument?
reply
Thou shalt not kill, that's all there is to it really

If you provide tools that help an organization kill easier, and possibly lazily rely on your inaccurate tools' judgment, and you do not care about the outcomes of these tools' uses, that's on your soul, too.

Yes, causality and attribution are hard sometimes, but it may have been that without it, hundreds of humans would be alive right now, and he doesn't even know. Maybe he should talk to the kids' parents about what alignment means.

reply
Ah, I didn't realize we're having a religous discussion. If all there is to it is "thou shalt not kill" then I guess that according to your POV he shouldn't have sold to the military even if the system was only suggesting "valid" targets.
reply