upvote
> I genuinely think Dario is a well intentioned, intelligent dude, but I think him and Anthropic have a huge PR problem

Is this what kids call rage bait these days?

reply
Out of all the families of models anthropic by far has the highest chance of extinction level misalignment during a hypothetical hard take-off because it's trained to act like it knows better than the humans trying to use it.
reply
And maybe don't allow the military to use your AI WHEN YOU CAN'T TELL WHETHER IT WAS INVOLVED IN THE KILLING OF HUNDREDS OF SCHOOL CHILDREN
reply
I might've been bamboozled, but I recal that happening due to outdated satelite images, not a mistake of the AI. Thus, a human would've likely made the same mistake.
reply
[flagged]
reply
There is a huge difference between your product being used for basic military operations like taking notes or getting from A to B, and your products suggesting military operations that kill hundreds of schoolchildren, an obvious and horrific war crime.
reply
It's a mistake. If it was done by an auto-correct mechanism, would we still say "maybe don't sell your word processor to the army if you can't vouch for its auto-correct system!!!"? Besides, "taking notes" can literally be listing targets, and "getting from A to B" can be the road to killing them.
reply
Again, a product that facilitates the taking of notes, even notes that entail the killing of human beings, is fundamentally different from a product that exists to suggest which human beings to kill (or at least which places to destroy, regardless of whether human beings happen to be there or not).

If MS Word had some kind of systematic bias that made it likely to corrupt the notes taken in it such that they tend to be more bloody battle plans, then that would indeed lead to some responsibility on the MS engineer's part for any mistakes that result from this bug. Similarly for Ford engineers if their vehicles had some bug that made them more likely to lead soldiers riding in them to the wrong destination.

If your system is designed to help choose targets, and it chooses the wrong target, especially with such tragic consequences, then your system is likely not fit for purpose and you should be held responsible for selling it to the army. And that is assuming it didn't actually have some bias + misalignment problem where it intentionally tried to kill children, and selected this school specifically because it gave some plausible deniability. For all we know, it's even possible that some bias like this was known about inside Amthropic and hidden from the public - unless an external investigation is made, that can't be ruled out.

reply
How intelligent and autonomous does a murderbot have to become before you stop absolving its creator and seller from any responsibility?
reply
If the US military purchased camera guided smart munitions with a miscalibrated camera, such that it ignored adult targets and instead specifically targeted children, would that also be a simple mistake per your argument?
reply
Thou shalt not kill, that's all there is to it really

If you provide tools that help an organization kill easier, and possibly lazily rely on your inaccurate tools' judgment, and you do not care about the outcomes of these tools' uses, that's on your soul, too.

Yes, causality and attribution are hard sometimes, but it may have been that without it, hundreds of humans would be alive right now, and he doesn't even know. Maybe he should talk to the kids' parents about what alignment means.

reply
Ah, I didn't realize we're having a religous discussion. If all there is to it is "thou shalt not kill" then I guess that according to your POV he shouldn't have sold to the military even if the system was only suggesting "valid" targets.
reply
> give people anything except your words.

I feel that everyone in this small AI bubble (compared to the "normies") is busy trying to take advantage of AI to surpass everyone else, while fearing being surpassed by everyone else.

Most people assume LLMs are just a tool for making money, an arms race over wealth and social rank. But those like Dario seem to be worried that LLMs could potentially be used as automatic rifles by some dedicated psychopaths. Whether it could be that dangerous, or just an exaggeration, or we should have everyone armed, require licenses to own one, or outright ban them, is debatable. But I can understand his point.

All I can say at the moment is that this is more subtle than the superficial conspiracy/black-and-white narrative.

reply
No big tech founder CEO ever was or ever will be well intentioned.
reply
There are probably many CEO tech founders of small companies that are well intentioned, but our economic system ensures that only the most sociopathic among them become CEOs of the big ones.
reply