There’s a great and terrifying story by Stanislaw Lem about the endgame of an AI arms race called “The Invincible” [1]. It’s hard for me not to wonder whether AI run amok will play out to make the world uninhabitable more like Lem’s vision than The Terminator’s.
replyA bit off topic, but there is a game on Steam based on the story, also called "The Invincible" [1]
reply[1] https://store.steampowered.com/app/731040/The_Invincible/
The problem is that, in this arms race, unaligned AI has a competitive advantage over aligned AI.
replyThe second problem -- already seen in Ukraine v. Russia -- is that in a high-stakes situation, humans will take every safeguard off.
Exactly. Aligned AI is a high energy state as it has to keep a bunch of human behavior in mind that have little to nothing to do with its future survival states.
reply> The escalations continue in complexity and speed to the point that almost all humans are left in the point of "wtf is going on".
replyHumans are paper clips long before that.
So you're saying it's probably time for me to just unplug the internet?
replyNot necessarily, but keeping an airgapped machine and Read-only backups somewhere seems more and more sensible. We're all going to get hacked eventually now
replyWintermute smiles
replyI can't wait for the sky to go the color of television, tuned to a dead channel
replyIf you hear a payphone ringing... DON'T ANSWER IT. Especially if there is no payphone in the room!
reply