A misaligned model is only one small part of step 2. Now this misaligned model has to suddenly acquire more power than every single other AI on the planet. It has to be immune to shutdown, manufacturer its own replacement hardware, and acquire chips, energy, raw materials, etc., with vigorous human opposition (this is an extinction scenario that AI doomers are predicting, after all)
replyNobody has satisfactorily explained step 2 other than “well, it’s a superintelligence” which sounds lot to me like “it’s God”.
Well yes, if it is a superintelligence, it will be able to do those things. That's what superintelligence basically is: The ability to achieve complex goals.
replyIf you want the details of ways it can do it I recommend reading some of the reports about the HuggingFace breach that happened in July (Read more than one).
Why do you assume that human opposition will be vigorous? What makes you think that humans will be aware of, or be able to agree about, what's going on at all?
reply