upvote
That's an easy reading, sure.

Unfortunately, many (most?) of them also seem to think they're better equipped than all of the others to do it safely, so "shut down my own company" effectively means "let one of the others destroy the world", whereas "keep company running" has at least a chance of "I solve alignment, we have a happy ending" (your case C).

Note: this does not mean I agree with them. Obviously they can't all be correct that they're safer than the others.

> If they believe A, why are they so incredibly sloppy about security? Why did they outsource part of it to some external firm instead of leveraging their own expertise on the technology?

The tech they're experts at is AI, not security, which answers both parts of that.

Outsourcing things you are bad at is normal, not a mystery. Lots of places value physical security, and therefore hire a private security firm.

reply
They can believe A and B and at the same time they don't want to cripple themselves. Your logic does not hold up.

Similar situation: nuclear race - everybody knew the risks, but making yourself armless does not help in any way. You need to make sure everyone is on the same page before you make yourself vulnerable in any way.

reply
If there's a company with an insanely high valuation, and two potential motives consistent with their actions, one altruistic and one greedy, I'm struggling to understand why the assumption would be altruism, rather than the overwhelmingly more common primary motivation of "we want to make a boatload of money" while finding it useful to pretend otherwise, especially when so many sizable investors have similar sizable expectations for their returns.
reply
I don't see the greedy path - why would stopping research - the biggest advantage frontier labs have, did bring any value to them?
reply
It seems to me the ROI on the research may be diminishing. If they aren't turning a profit with the infrastructure already deployed and the models already trained then buying more GPUs and spending billions more on training doesn't seem like it would improve the correct side of the balance sheet.
reply
My problem with the “making everyone more productive” is that it’s a feel-good propaganda for the individual. Most capitalist see people as an expensive cost to be eliminated. Thus all the company shares going up when they announce layoffs.

The market can only absorb so much new products, so if productivity increase, companies will reduce headcount as much as possible to increase margin for the same income, not grow their product or production to make use of their staff.

(See any wage/productivity graph)

Government will also likely follow the same path of reducing headcount instead of producing better / faster outcome for their citizens (except the internal surveillance apparatus. The one never shrinks).

reply
Reducing the cost of a desirable service or product often has a tendency to increased demand for that service or product, and means customers have more capital available to purchase other products and services increasing demand for those. These factors are significant drivers of overall economic growth.

Of course economic growth has it's own problems in terms of ecological impact and such, but if you want to reduce ecological impact you still need to improve productivity. It's a matter of how you spend that efficiency improvement as a society.

reply
Yeah, I'm not convinced either. I'm just saying that's what I think the AI labs believe.
reply
> actually wanting to exterminate humanity is too comically supervillain esque

An entirely plausible scenario is that these oligarchs dream of living in Solaria, the planet from the Asimov universe where only a small number of immensely rich people lived and all the work was done by hordes of robots.

Once humans no longer serve the needs of the oligarchs, why would they want billions of people around? A question to ponder.

reply
Whole point is that billiinares already have whatever they want without any meaningul control from society.

Humans serve them well enough and relatively easy to control. Robot utopia is not guatanteed and have own risks.

And fortunatelly in a group of AI overlords everyone except of Musk is pretty sane.

reply
If they were rational people they'd likely have stopped their quest after the first few millions. They are driven by pure greed and their humongous egos, none of them is sane.
reply
I dont think there is anything wrong with people driven by pure greed and wealth maxxing.

They are far safer people compared to those who dont care about virtual wealth numbers.

reply
Some of it might be that, but some of it might be more prosaically the same thing that keeps people playing competitive games on one hand, or cookie clicker on the other. (Cookie clicker could read as "pure greed" if you squint.)
reply
Yes, but me playing cookie clicker (or fortnite, or whatever, competitive tic-tac-toe) does not bring with itself suffering and poverty for millions of others. Whereas them chasing their virtual money high score ruins societies.

They are all psychopaths, devoid of normal human emotions.

reply
>Whole point is that billiinares already have whatever they want without any meaningul control from society.

Or maybe they don't. Some of those are deeply afraid of society looking funny at them.

reply
> Humans serve them well enough

For now. Throw in climate change, food shortages, war, mass migration, and suddenly the ratio becomes 25 million : 1 for starving, angry people vs billionaires, globally.

reply
Just a few tens of thousands to entertain them, including kids for their "islands", would be enough.
reply

    "I propose something different. Listen, my dear enemy... I shall acquire absolute power on earth. Not a single chimney will smoke unless I order it, not a single ship will leave harbour, not a single hammer will strike. Everything will be subordinated — up to and including the right to breathe — to the centre, and I am in the centre. Everything belongs to me. I shall engrave my profile on one side of little metal discs — with my beard and wearing a crown — and on the other side the profile of Madame Lamolle. Then I shall select the 'first thousand,' let us call them, although there will be something like two or three million pairs. They will be the patricians. They will devote themselves to the higher enjoyments and to creative activities. Taking an example from ancient Sparta we shall establish a special regimen for them so that they do not degenerate into alcoholics and impotents. Then we shall determine the exact number of hands necessary to give full service to the culture. In this case, too, we shall resort to selection. These we shall call, for the sake of politeness, the toilers—"

    "It goes without saying—"

    "You may laugh, my friend, when we get to the end of this conversation... They will not revolt, oh, no, my dear comrade. The possibility of revolution will be destroyed at the very root. A minor operation will be carried out on every toiler after he has qualified at some skill and before he is issued a labour pass. Quite an unnoticeable operation made under almost accidental anaesthesia... Just a small perforation of the skull. He will get a bit dizzy and when he wakes up he will be a slave. Lastly, there will be a special group that we shall isolate on a beautiful island for breeding purposes. All those left over we shall have to get rid of as useless.

    "There you have the structure of the future mankind according to Pyotr Garin. The toilers will toil and serve uncomplainingly, like horses, for their food. They will no longer be people and they will have no worries except hunger. They will find happiness in the digestion of their food. The elite, the patricians, they will be demigods. Although I despise people altogether, it is always pleasant to be in good company. I assure you, my friend, we shall enjoy the golden age the poets dream of. The impression of the horrors created by purging the earth of its surplus population will soon be forgotten. On the other hand, what opportunities for a genius!

    "The earth will become the Garden of Eden. Births will be regulated. There will be selection of the fittest. There will be no struggle for existence, that will be lost in the haze of the barbaric past. A beautiful and refined race will develop, with new organs of thought and sensation. Communism trying to drag all of humanity to the heights of culture? I instead will do it in ten years... What the hell! In less than ten years! For only a few, true... But then, it is not a question of numbers."

    "A fascist Utopia, rather curious," said Shelga. "Have you told Rolling anything about this?"

    "It is not a Utopia, that's the funny part of it. I am only logical."
Well, what was considered reasonable enough to state outright in 1906 or in 1926, is not quite polite to say out loud in 2026, but I don't think the sentiment of contempt has ever quite gone away entirely.
reply
From https://en.wikipedia.org/wiki/The_Garin_Death_Ray if anyone else was wondering.
reply
It's not clear to me that what the evidence for D is. It could just as easily be that they want to translate popular fear/distrust of AI and desire for regulation into something anticompetitive that benefits them over the smaller players rather than being regulated themselves.
reply
There are some nasty game theory outcomes to keep in mind here. (See also : cold war nuclear arms race.)

https://www.lesswrong.com/posts/hc4DbmhdzZpSLMQ9Y/the-ai-rac...

reply