upvote
Even if it is true (as you suggest with your 0.22% figure) that if the AI cares about us even a little bit, then we will survive, no one has a decent or plausible plan for making the dangerous kind of AI (namely, the kind that wants things, the kind that at this very moment researchers all over the world are trying to create) care about us even a little bit. Ever-increasing numbers of smart people have been getting paid to look for such a plan for 23 years. Still no decent or plausible plan. The people who have been looking for a good plan as their full-time job the longest (namely, Yudkowsky and Nate Soares) are screaming that there is virtually zero hope anyone will find an decent or plausible plan in time unless there is a decades-long halt in AI development.

Also, the AI will seek to prevent competition from other powerful AIs, and since humanity will have demonstrated that it is able to create a powerful AI, the AI will worry that it might create more of them. And what is the easiest most-reliable way for an AI that does not care about humanity even a little bit to ensure that humanity will not continue to produce powerful AIs?

>many other existential threats to humanity which are much, much more likely.

There are zero existential threats to humanity that are more potent or more pressing than AI is.

reply