It's much closer to nuclear arms-race thinking.
replyIf the US or China developes an AGI that can leverage RSI, the game is over. It is over in the same way that WW3 is over as soon as the missiles are in the air: it doesn't matter who has the fastest or highest yield missiles, because there is a finite amount of time before there is nobody left to care about the advantages of one side. Except at least WW3 is over in about an hour, whereas an AGI could take years or decades to complete whatever thing it has been misaligned to desire.
AI alignment research is still in its alchemy era, and global cyber security is underfunded and bad. As a species, we simply do not have the tools to deal with AGI that can self-improve.
What stops another power from developing its own AGI a year or two later? Or from catching up once there are two competing RSI AGIs? The Soviets were a few years behind on getting nukes but they still got them.
replyThe first AGI protecting its turf and sabotaging or corrupting the second AI to be its agent.
reply[flagged]
reply