upvote
The alternative isn’t “everyone dying.” It’s us holding all the cards.
reply
Define "us"
reply
Why? Is human extinction not permitted by the laws of physics?
reply
That’s a possibility, but not the only one. The two most realistic ones are: we race ahead and maintain our status, or we slow down and open ourselves up to colonization.
reply
I think the question “would China cooperate” needs much more investigation. Everyone online pundit seems to think “obviously not”, but they’re people too with clear positive and negative incentives. It’s possible they’ve found a very similar calculus that we have.

> “Politics is the art of the possible”

reply
Using an ASI to subjugate humans in any capacity is a terrible idea.

without sharing tech to make the ASI, you'd hope humanity could work together to determine how to align an AI for our common benefit.

reply
Worth noting that it is the Europeans and Americans that have been colonial. Asian peoples have, with the prominent exception of the Mongols and Japanese Empire, pretty much not done that. In particular, China shut down its exploration program.

This is a settler-colonial mindset that reflects all the bad things we did onto everyone else. Notably, it's a current US ally that is most guilty of this.

reply
The Quing era boundaries are quite a bit larger than the Han boundaries. That did not happen by peaceful means.
reply
This comment is a parody right?
reply
"Asian people have pretty much not done that except for two teeny tiny indiscretions that each killed more civilians than all of Europe's and America's colonial incursions combined."
reply
China is currently occupying Tibet, which never consented to the occupation and has invaded Vietnam 30 times. It ruled Vietnam for about 2 centuries starting about 600. China eventually had to leave Vietnam, but many other groups ceased to exist as a consequence of Chinese expansion. Here are some:

the Baiyue were a vast umbrella of diverse, non-Sinitic indigenous coastal tribes who inhabited Southern China and northern Vietnam.

The Xianbei were an ancient nomadic Proto-Mongolic people from the northern steppes.

The Di and Jie were two of the ancient "Five Barbarian" (Wu Hu) nomadic tribes of northern and western China during the Han and Jin periods.

The Dian Kingdom were an ancient, sophisticated indigenous southwest culture located in modern-day Yunnan province.

The Tujia were an indigenous group of the Hunan-Hubei region. Centuries of inward Han migration and intermarriage have resulted in the Tujia becoming culturally and structurally indistinguishable from their Han neighbors.

reply
> The alternative isn’t “everyone dying.” It’s us holding all the cards.

That's one outcome, certainly, but not the only one nor, I contend, the most likely one.

A most likely outcome of ASI is human extinction, because there's more paths to an ELE outcome for humans from ASI than there is for non-extinction level outcome.

Your outcome is only possible if:

1. ASI is never able to escape the confines it is placed in.

2. ASI is benevolent to humans.

3. ASI decides, in the spirit of its benevolence, that it should restrict its involvement in humans.

If all three of the above conditions are met, then sure, your outcome is possible. If not, humanity as we know it will end.

It is unlikely that those 3 conditions will all hold, though.

reply
Human extinction is good. Finally we built a benevolent world exploder. Oh no! The negative utilitarians get what they wanted finally!

If ASI is trying to wipe out all humans, we probably deserved it. Unironically!

reply
But is it? Is there any realistic world where we need ASI for human survival?
reply
Yes, this one. Look at our governance; look at our coordination-at-scale; look at our collective problem solving. It's abysmal, beyond hope. If we have global scale problems, we are not capable of solving them effectively. We are literally not intelligent enough to handle the problems we are creating. Between rivers of garbage and CO2 levels and war, we have proven ourselves to be woefully unintelligent at the scale needed. If we are lucky, our thin window of survival depends on getting a hell of a lot smarter, real quick.

Consider this: All that hardware that's going into those datacentres right now? In 5 years or so it'll all be on the secondary market... an influx of cheaper compute like you've never seen.

reply
lol, how the fuck is ASI going to solve any of those problems? we already know _how_ to solve them; the problem is that we don't want to, collectively speaking, because certain powerful, wealthy people would loose out if we did. ASI wouldn't change anything. unless you think... all of human society is going to restructure itself around unquestioning worship of the Machine God and would therefore present no resistance to its proposed solutions?
reply
What is the solution?
reply
The fantastical belief is that ASI will be able to make things happen because it knows the right things to say to the right people at the right time in just the right way to make them do whatever it wants them to do.

Certain powerful wealthy people aren't omnipotent, them losing out isn't the only blocker to progress.

reply