upvote
Some of the most reassuring and scariest things you can read are about the incidents that have already occurred where computers said "launch all the nukes" and the humans refused. On the one hand, good news! We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. Bad news, it's been skin-of-our-teeth multiple times already.

https://www.warhistoryonline.com/cold-war/refused-to-launch-... - This isn't even the incident I was searching for to reference! This one was news to me.

https://en.wikipedia.org/wiki/Stanislav_Petrov#Incident - This is the one I was looking for.

reply
> We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to.

previously no-one had spent trillions of dollars trying to convince the world that those computers were "Artificial Intelligence"

reply
They had to do with "state-of-the-art radars", "military-grade communication systems", etc.
reply
Yeah but they dealt with sota and military-spec systems their entire career and they know that it just means "lowest bidder".
reply
Or "alignment" which means "let's ensure the AIs recommend launching nukes only when it makes sense to, based on our [assumed objective] values"
reply
of course they did. That's the literal topic of War Games (1983). You should actually be somewhat reassured that we aren't living during the era of Dr. Strangelove where you had characters in the military industrial complex who were significantly more insane when it came to the beliefs of what computer systems and nukes can do.

There was a time when people wanted to dig tunnels with nukes https://en.wikipedia.org/wiki/Project_Plowshare

reply
> There was a time when people wanted to dig tunnels with nukes

The article seems to be about mining rather than tunnelling.

And the issue with the idea being? We also dig using explosives, there isn't an in-principle problem. Reading the wiki article it looks like the yields were excessive, but at the end of the day mining involves the use of things that go boom. It is easy to imagine small nukes having a place in the industry.

reply
Digging tunnels with nukes sounds better to me than shooting them at each other!
reply
> We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to.

This relies on processes being in place to ensure that a human will always make the final decision. What about when that gets taken away?

reply
I find it hard to imagine that the people in a position to kill those processes could ever be that zealously in love with AI, but recent events have given me a tiny bit of doubt.
reply
I briefly got into a "rabbithole" of watching videos about trying to intercept BMs and glide hypersonic weapons, pretty interesting, decoys deployed in space... the outcome seemed to be not good, can't guarantee 100% interception
reply
A missile will always be cheaper than a missile interceptor, and the interceptor will never be a 1:1 kill. Building a missile interceptor system ia a good way to get your strategic opponent to build a bigger stockpile.
reply
Disagree on always being cheaper. Military planners are obsessed with the best weapons, such interceptors are pricey. But look at Israel: Iron Dome. ~$50k/shot. They deliberately built a dumb SAM because it was designed to go against dumb opponents--objects falling freely on a ballistic trajectory. While they are usually facing light stuff that isn't even worth that they have successfully engaged longer range stuff that costs many times what the interceptor costs.

Overall, though, the offense always wins this one because interceptors can only protect a limited area whereas missiles can go anywhere.

reply
Iron Dome is a great example of my point. It is a $50k interceptor designed to take out a propane tank with a rocket strapped to it, not a real ballistic missile like a Scud.

Patriot missiles ($7MM) take out Scuds ($3MM).

reply
We shouldn't be the least bit surprised no human has complied so far.

If they had, then we wouldn't be having this conversation. For all we know, there may be a vast multiverse of universes some with humans and we would only find ourselves having this conversation in one of the universes where no human pressed the button.

reply
By that logic, it may actually be pretty common for rabbits to swallow the sun. We just haven't seen it happen because we're in the wrong universe and would've died it it happened in ours.
reply
Anthropic Principle
reply
I hope humans in charge are as wise now as they were then.
reply
Surely that’s the definition of a quixotic hope.
reply
I am scared of two things.

First, people being rubber stamps for AI recommendations. And yes, it is not unreasonable that in a dire situation, someone will outsource their judgment (day).

Second, someone at the Pentagon connecting the red button to OpenClaw. "You are right, firing nukes was my mistake. Would you like to learn more facts about nukes before you evaporate?"

reply
If you think humans are going to delegate reasoning and responsibility to something, shouldn’t you also be concerned about the sorts of recommendations that thing is going to make?
reply
If you found out the pentagon was using a magic 8 ball to make important war decisions what would you want to fix - our military leadership or the inner workings of the toy?
reply
One of those sounds a lot easier than the other. The magic 8 ball toy company would also probably be pretty incentivized to not die in a nuclear holocaust.
reply
Unless you're suggesting the toy company secretly rigs the magic 8 ball to never recommend nuclear war, I'll take my chances with the organizational changes.
reply
That is indeed what I think the gp is suggesting I feel. And why not?
reply
Because if your leadership is stupid enough to trust the 8 ball they should not be in charge??!
reply
One can try themself, for Claude is fine at waging war [1]. Notice the thoughtful UX, including the typing "I ACCEPT FULL RESPONSIBILITY".

[1]: https://nitter.poast.org/elder_plinius/status/20264475874910...

reply
deleted
reply
The speed with which my technical cow-orkers and friends have started relying on the "AI Overview" only, in lieu of following any links, in search engine results (let alone not using search engines at all over chatbots) tells me reasoning and responsibility will be outsourced as soon as possible.

Humans are fundamentally lazy. The brain is an "expensive" organ to use.

reply
Trump's Golden Dome is literally advertised to help the U.S. win a nuclear war by leveraging AI.
reply
Elon's involvement in the nuclear military complex https://www.mintpressnews.com/pentagon-recruiting-elon-musk-...
reply