upvote
Not to be too pessimistic, but it strikes me that in a worst case scenario, severing a cable might scale to severing global digital communications which is itself a massive worst case scenario even if we removed AI from the picture.

If the worst case outcome is we have to even temporarily shut down global shipping, banking, transport, health services, communications and national infrastructure to contain a self-propagating misaligned AI that can evolve itself and work its way into any sufficiently large computer system to escape humans preventing it from finishing its task, that seems like something we should be trying very hard to avoid.

reply
deleted
reply
Or you could argue that the super intelligence will turn us into a pet of some kind. Then the risk is we think we are controlling it while we are really the super intelligence's lap dog.

That is an argument that at least makes sense.

The super intelligence as homicidal maniac just doesn't make sense. There is less intelligent wildlife all around us and we mostly completely ignore it. We have more interesting things to do with our time as intelligent beings than carry out a bird or rabbit genocide.

It almost seems like the projection of some kind of paranoid delusion about change.

reply
deleted
reply