upvote
yeah totally bro, that's why your Glock is doing half of your job right now.

Tip: If someone was spending billions of dollars figuring out how to equip your Glock to stand up, unlock a door, walk around, talk to people, and make decisions... it would be dangerous.

reply
General intelligence has very different adversarial dynamics compared to nuclear weapons.

Explained further here: https://news.ycombinator.com/item?id=49094348

Though there is further nuance in agentic vs non-agentic AI: https://news.ycombinator.com/item?id=49736149

reply
I'm legitimately interested in this argument, but simply stating that good vs evil are in equilibrium is nowhere near enough.

> There will be new adversarial games when everyone's smarter (bio/cyber offense/defense), but those games are always symmetrical in the long run

This is an article of faith, not an argument.

reply
The continuation of the sentence you quoted is "and there will also be new cooperative strategies."

This is not faith, but an observation of Earth's past. If it were the case that intelligence is itself harmful, then it should not have evolved in so many species.

And even if we look at only humans, we do not see dumber populations being more prosperous. One could argue about the definition of "dumber" and "prosperous", but it is at least true in my own judgement of value, which I suspect is not too far from the average in modern society. One could also argue that bio/cyber adversarial games are qualitatively different from all past adversarial games, but you could argue the same thing for other qualitatively different games that emerged in the past, when they were new. But we're still here. So this argument hinges on burden of proof, and I think that's on Anthropic.

Also, as mentioned in that first linked comment, centralizing AI development increases a different kind of speciation risk: a small group of superintelligent humans, likely the ones currently running the top AI companies, splitting from the rest of humanity.

reply
The only danger here is allowing them to monopolize it. We need to secure the same capabilities for ourselves.
reply
I agree! Same with thermonuclear weapons and Glocks-on-autonomous-Segways!
reply