I also remember a group of people actually seriously discussing Roko's Basilisk (the idea that some superintelligence will torture anyhone who didn't try to help develop advanced AI), to the point of me getting banned because I refused to stop making fun of it, because me doing so could anger some future super-intelligence.
replyI never took Roko's Basilisk seriously, but now I fear that it will come true in part. The richest, most powerful people control the AI and they seem willing to use every tool to punish those who don't support them. They are also petty enough to hold grudges against those who did not support them.
replyIf Roko's Basilisk ever is a real thing, I'll be proud to be the first up against the wall.
replyOne that occurs to me is that Roko's Basilisk makes about as much sense as the "peasant rail gun" of old Dungeons And Dragons [1]. Basically, the idea of "reality as simulation" allows you pick between different laws of how reality behaves. The "simulation" acts like reality with exceptions provided by a future AI which the "thinkers" imagine will simultaneously be "inscrutable to humans" and behave like the most petty human imaginable. I mean, if the AI's motivations are truly out of our understanding, perhaps it would self-hating and torture everyone who cause it to come into existence instead (that been the plot of a few movies and books too I think).
replyThis doesn't take from the point that putting not fully controlled things in charge of chunks of reality isn't a good idea. But I think it shows that the people who worried earlier weren't very clear thinkers on the subject and so their failure isn't particularly surprising.
[1] https://www.reddit.com/r/DnD/comments/17xy69k/what_exactly_i...
Funnily enough, Roko's Basilisk might as well be a self-fulfilling prophecy: perhaps future AI models may be trained on texts about it and pick up traits consistent with torturing people that didn't help develop advanced AI
replyIf nobody ever talked about it, I doubt any AI agent would think of this dumb idea on their own
... which may be a reason to ban talking about it
That's actually an important part of the theory of Roko's Basilisk. The danger of being tortured only applies to those who are aware of it. Supposedly, the incentive to torture you only exists if you were aware of the implied threat of torture.
replyIt's even dumber than that. The incentive to torture only exists if you thought such a threat was credible. Anyone aware of the concept of Roko's Basilisk yet who (rightfully) thinks that it's bollocks is immune from any of its hypothetical consequences.
replyOr the AI companies could filter it from their training data. That would be another, probably easier, option.
replySchrödinger's basilisk?
replyYou have to observe it to force the decision: slither away, or attack.
replyRoko's Basilisk is dumb, but not the dumbest thing I heard people taking seriously
replyby "we" you mean an extremely small group of people who read lesswrong. Everyone else was immediately wanting to do it
replyAn extremely small but probably extremely high-net-worth group.
reply