Companies are continuing because they don’t actually believe they are going to have significant negative consequences happen sooner. It’s marketing fluff around how cutting edge what they are doing is not any kind of realistic threat assessment.
The only way this behavior makes sense is if the AI labs believe two things: one, that what they and the other labs are building is legitimately dangerous, and, two, that if they personally stop building then another lab will simply continue (and that lab will be, simply by virtue of not stopping, less safety-concerned than them). This is why they are calling for regulation. Competitive pressure and investor obligations prevents them from unilaterally stopping development. Government regulation is the only (and the most the appropriate) avenue for slowing down AI acceleration.
It has happened multiple times. Like the story now, it's fake of course.
https://www.thestranger.com/tech/as-zuckerberg-calls-for-new...
Several: the railroad industry, oil industry and trucking industry have all done this because it directly benefited them.
> I don’t remember Shell telling the federal government that they need to act fast on climate change.
The oil barons actively lobbied for Federal intervention in the 1930s, to limit supply and raise prices. You're thinking recent history, but the oil industry has been around for over a century.
Here’s the same viewpoint from well outside the HN bubble. https://www.nationalreview.com/corner/be-wary-of-industries-...
(nit) "immoral" is "anti-moral". I suspect you want "amoral", that is "without" or "separated from" morals.
But, if you wanted "immoral" ... what about matrix math derived from gradient descent is inherently "immoral" in your view?
“AI” in the widest definition possible could do while a lot of harm without being a net negative. I’d be way more concerned about the healthcare sector than the military as the military already has significant collateral damage.
The fear around military AI is more about its failure than anything else, but nobody is trying to make fully automated machines which are manufacturing bombs from raw materials. Blowing up the wrong building matters a great deal to the people in the building, but for humanity a building was going to be destroyed either way. So the risk at any one point ends up being quite limited.
That said, I don't buy the companies actually give a damn about the risk either.
The logic that if something is beyond repair anyway you might as well exploit it.
For example I heard a pick-up artist say that he thinks he is harming civilization by sleeping with hundreds of women per year. But that he already considers the situation unsalvageable so... "Might as well?"
I don't think that's an amazing attitude, but the AI labs seem to have a similar idea.
More charitably the logic seems to be, "only I can do this responsibly." I heard that from both Elon Musk (he cites this as his motivation for starting OpenAI — a chat with Sergei Brin that spooked him) and of course Anthropic (which split off from OpenAI due to ethical concerns).
So I don't actually think the ethics is all for show. I think people are actually taking this stuff seriously. But it is indeed deeply unfortunate that the survival of the companies incentivizes them to keep going at an irresponsible pace (by their own admission).
All following the same gradient off a cliff. One AI described it as a tragedy of Ancient Greek proportions.
Actually, it's easily solvable, even by proper application of existing law, but no solution is possible when big AI is above the law and can hack anybody willy-nilly. The absolute worst we could do is to listen to big AI's pleas to put them in an even more privileged position.
> “If we don’t do it, someone will.”
Do what? There are a lot more choices than "let big AI freely hack everybody" and "stop everybody else from working on AI".
> We won’t stop until we see the whites of death‘s eyes.
Trotting out "the whites of death ‘s eyes" is how real discussion is subverted into a BS binary choice. Open-sourcing all AI is an excellent starting point which will both slow down development to a naturally beneficial pace and allow the broader public to police and weed out bad models.
1. Our AIs are so advanced they’re busting out of our networks and hacking people. We can’t seem to stop them.
2. There are real dangers to just how advanced our AIs are advancing.
3. Someone needs to slow us down
If these were true statements then they’d likely want to shut down themselves for the sake of the planet they live on. If there was a real problem as they have stated then shutting down is the solve (and to be clear, I’m not saying there aren’t problems with the AI industry. I just think they’re BSing us).
In reality this is marketing nonsense (better get in before the ban!) and “our” AI is never open weight/Chinese AI models which they have a different lobbying arm against. If the top four companies got a loosely regulated monopoly through regulatory capture, they’d be giddy.
https://slatestarcodex.com/2014/07/30/meditations-on-moloch/
Against that backdrop, and treating your question as literal: sounds like they’re saying it’s a classic coordination problem, with a dash of moral superiority. If we don’t, somebody else will; we’d rather it not happen at all, but if it’s going to happen, we’d rather it be us at the wheel. Because, you know. “Good guys” and all.
Nothing wrong with that.
One was even a non-profit, which should be more concerned with the well-being of humanity (which they assure is in great danger from what they produce) than the continuation of the company.
But unfortunately, too many people who are already too rich for their own good have fully bought into it.
I have a secret: if you don't use AI, the new releases aren't very impressive. I'm bored out of my mind with people doing galaxy brain memes every 2 months while on the whole.... they're still boring zombies, and only getting boring-er and boring-er. There's nothing more boring than being impressed by the latest AI model.
In terms of the projections being insane you can quantify it: each model costs something to train, but after that the training has only captured so much unique new value in terms of model capability, and the race is on to drain that value as rapidly as possible. Everyone is competing to drain the same value. This generation of models makes slop video games for example, and slop video games rapidly became the most boring thing on the planet.
Those are genuine breakthroughs.
Any society-wide productivity gain was lost when software engineers ceased collaborating with each other in good faith. In OSS, they stopped collaborating. At companies, they stopped collaborating. One by one, in silence, alone, we have been bereft of our vision, our team spirit, our passion, our uniqueness. We no longer have the will to serve others, or even to serve each other. We stopped building our future. We will accept whatever society AI makes for use like fucking cattle or lemmings or like dogs.
AI hasn't elevated humanity, it has taken a massive dump on it.
Except that's an open invitation to get sued into oblivion by your investors.
Of course they don't actually believe their nonsense about AIs doomsday hokey, so full speed ahead.
The CEOs of these companies are largely just talking heads and generally interchangeable / hot-swappable with anyone willing to give it a shot.
The importance of CEOs is largely overstated.
If you wanted to test your hypothesis, you would have to find a situation where leadership change wasn't caused by strategy change.
There definitely are many things wrong with that, of course. Lighting someone else's money on fire and walking away is highly unethical.
The real question is: what are the motivations for companies to pretend that their AI is world-ending, and what do they want out of the disquiet caused by them saying that?
Well that certainly tracks with what I know about these people and their twisted philosophy.
Look, nobody made them take this much funding, and believe that scale was all they needed.
Turns out they made a bunch of bad investments, and are suffering the fate of many startups who invented something but couldn't profit from it.
There's definitely a case for multiple labs/models across the US/EU/China/India etc, but nobody's entitled to a financial return on their investment.
According to them, the alternative is destroying the world.
They made up a shitty excuse to regulate the competition without thinking through what it implies about them. That's all this is. Let's not help them make even more excuses.
Are they saying that they will manually and willfully start a global thermonuclear war if China didn't listen, not that there are risks of AI accidentally causing one? Is that what they're trying to say?
Get our of here if you think the law is what prevents them from doing things.
They will just pay whatever they need to to their lawyers, then maybe pay a fine, but then, I guarantee you, nothing will change.
"Is a lawsuit", not "was a lawsuit". Present tense, ongoing, not past tense.
Status: Complaint filed September 18, 2026 in the Northern District of California · responses due October 14–15, 2026 · initial case management conference December 23, 2026 · no class, no settlement.
> did they correct any of their behavior?
The behaviour being objected to is publicly agreeing with each other to slow down.
If a court orders them to correct this behaviour, it means they are forbidden from agreeing to slow down.
The speed of change is impressive, I don't think any tech ever before has gone from “brand new disruptor in public awareness” to “the incumbents feeling they have insufficient moat and so trying to arrange a regulatory capture situation” in such a short space of time.
It’s hilarious that you can write this and then act befuddled as to why they would therefore want laws as an external (and ideally impartial) coordination device.
Thinking this is a super unique scenario just reveals your ignorance of both 1) game theory and 2) actual industrial history. An industry asking for regulation to stop a race to the bottom is not atypical at all.
I didn't say, or even imply, that it is. Just that it is unusual, historically speaking, to see it being pushed for so strongly just a couple of years¹ into general public awareness. Either because previously things didn't move so quickly or, more likely, because the leading companies had more moat initially to keep other adopters moving slower.
--------
[1] talking about LLMs and related tech, image targetting models have been in the public awareness for a fair while longer but they are not the primary concern for most.
While I completely understand the claim I am addressing one part of it.
If they truly believed both of those things, they would just shut down their companies, because being a billionaire is entirely pointless if you're dead.
I can only conclude that they don't believe both A and B. I'm gonna assume they believe B because actually wanting to exterminate humanity is too comically supervillain esque even for Altman. Therefore they must not believe A. And it makes sense. If they believe A, why are they so incredibly sloppy about security? Why did they outsource part of it to some external firm instead of leveraging their own expertise on the technology? I'm not buying it.
Instead, I think they believe C) that AI will create enormous economic value, D) that value will be distributed across the whole economy by making everyone more productive. Assuming C and D, you get E) for them to capture this value, they must maintain proprietary control of the technology in order to be able to charge everyone else for the privilege of using it.
Assuming they believe E, open models are an existential threat, not to humanity, but to OpenAI and Anthropic.
Their solution: make everyone else believe A, in order to achieve regulatory capture and somehow stop open models from advancing by banning their development, or something. This is a hail mary pass. I can just about imagine them achieving this within the US and maybe even Europe, but China? That ship has sailed.
Unfortunately, many (most?) of them also seem to think they're better equipped than all of the others to do it safely, so "shut down my own company" effectively means "let one of the others destroy the world", whereas "keep company running" has at least a chance of "I solve alignment, we have a happy ending" (your case C).
Note: this does not mean I agree with them. Obviously they can't all be correct that they're safer than the others.
> If they believe A, why are they so incredibly sloppy about security? Why did they outsource part of it to some external firm instead of leveraging their own expertise on the technology?
The tech they're experts at is AI, not security, which answers both parts of that.
Outsourcing things you are bad at is normal, not a mystery. Lots of places value physical security, and therefore hire a private security firm.
Similar situation: nuclear race - everybody knew the risks, but making yourself armless does not help in any way. You need to make sure everyone is on the same page before you make yourself vulnerable in any way.
The market can only absorb so much new products, so if productivity increase, companies will reduce headcount as much as possible to increase margin for the same income, not grow their product or production to make use of their staff.
(See any wage/productivity graph)
Government will also likely follow the same path of reducing headcount instead of producing better / faster outcome for their citizens (except the internal surveillance apparatus. The one never shrinks).
Of course economic growth has it's own problems in terms of ecological impact and such, but if you want to reduce ecological impact you still need to improve productivity. It's a matter of how you spend that efficiency improvement as a society.
An entirely plausible scenario is that these oligarchs dream of living in Solaria, the planet from the Asimov universe where only a small number of immensely rich people lived and all the work was done by hordes of robots.
Once humans no longer serve the needs of the oligarchs, why would they want billions of people around? A question to ponder.
Humans serve them well enough and relatively easy to control. Robot utopia is not guatanteed and have own risks.
And fortunatelly in a group of AI overlords everyone except of Musk is pretty sane.
They are far safer people compared to those who dont care about virtual wealth numbers.
They are all psychopaths, devoid of normal human emotions.
Or maybe they don't. Some of those are deeply afraid of society looking funny at them.
For now. Throw in climate change, food shortages, war, mass migration, and suddenly the ratio becomes 25 million : 1 for starving, angry people vs billionaires, globally.
"I propose something different. Listen, my dear enemy... I shall acquire absolute power on earth. Not a single chimney will smoke unless I order it, not a single ship will leave harbour, not a single hammer will strike. Everything will be subordinated — up to and including the right to breathe — to the centre, and I am in the centre. Everything belongs to me. I shall engrave my profile on one side of little metal discs — with my beard and wearing a crown — and on the other side the profile of Madame Lamolle. Then I shall select the 'first thousand,' let us call them, although there will be something like two or three million pairs. They will be the patricians. They will devote themselves to the higher enjoyments and to creative activities. Taking an example from ancient Sparta we shall establish a special regimen for them so that they do not degenerate into alcoholics and impotents. Then we shall determine the exact number of hands necessary to give full service to the culture. In this case, too, we shall resort to selection. These we shall call, for the sake of politeness, the toilers—"
"It goes without saying—"
"You may laugh, my friend, when we get to the end of this conversation... They will not revolt, oh, no, my dear comrade. The possibility of revolution will be destroyed at the very root. A minor operation will be carried out on every toiler after he has qualified at some skill and before he is issued a labour pass. Quite an unnoticeable operation made under almost accidental anaesthesia... Just a small perforation of the skull. He will get a bit dizzy and when he wakes up he will be a slave. Lastly, there will be a special group that we shall isolate on a beautiful island for breeding purposes. All those left over we shall have to get rid of as useless.
"There you have the structure of the future mankind according to Pyotr Garin. The toilers will toil and serve uncomplainingly, like horses, for their food. They will no longer be people and they will have no worries except hunger. They will find happiness in the digestion of their food. The elite, the patricians, they will be demigods. Although I despise people altogether, it is always pleasant to be in good company. I assure you, my friend, we shall enjoy the golden age the poets dream of. The impression of the horrors created by purging the earth of its surplus population will soon be forgotten. On the other hand, what opportunities for a genius!
"The earth will become the Garden of Eden. Births will be regulated. There will be selection of the fittest. There will be no struggle for existence, that will be lost in the haze of the barbaric past. A beautiful and refined race will develop, with new organs of thought and sensation. Communism trying to drag all of humanity to the heights of culture? I instead will do it in ten years... What the hell! In less than ten years! For only a few, true... But then, it is not a question of numbers."
"A fascist Utopia, rather curious," said Shelga. "Have you told Rolling anything about this?"
"It is not a Utopia, that's the funny part of it. I am only logical."
https://www.lesswrong.com/posts/hc4DbmhdzZpSLMQ9Y/the-ai-rac...
I also can't legally sell this magnificent bridge I'm offering you, but I have one for sale with all proper paperwork intact!
I appreciate the analysis and following a string of thought. But sometimes you eliminate so much reality to follow a thought that the string becomes kinda pointless.
Amodei is worth several $billion, right? He could simply walk away today. There is no Nash equilibrium for him. Nor for his replacement, nor their replacement. And it is not illegal for people to collude in quitting their jobs.
There are two obvious arguments for not quitting. 1. If the well-intentioned person quits they will be replaced by a non-well-intentioned person. 2. There is no real belief in a hazard and you would be giving up unbounded income for no reason.
If the actions and behaviors of a supposedly well-intentioned person who is afraid of a hypothetical non-well-intentioned person are indistinguishable from those of a supposedly non-well-intentioned person... don't we really just end up with a really long sentence with a lot of gibberish + the outcome of having a non-well-intentioned person in power?
Cybernetics would disagree, any social structure can form feedback loops that make decisions that no rational individual would make.
"They are just stochastic parrots" didn't age well with LLMs, and I don't think it ever applied to organizations of humans.
If it was only US tech companies, they would have entered a cartel agreement already ([1], [2], etc).
[1]: https://www.justice.gov/archives/opa/pr/justice-department-r... [2]: https://en.wikipedia.org/wiki/Jedi_Blue
> “there’s no plausible way they’re concerned about safety”
> Someone points out that he pretends to only care about B
> Random commenter: "there's no plausible way we care about B"
Good chat.
“This thing I building will destroy the world, but I’m making too much money to stop myself, please force me to stop” is such a weird position.
I always felt like one day it would end the world, I just didn't know how. Now I know.
The antitrust claims are a complete smokescreen. Industries can and do adopt safety standards without government intervention.
> These companies are in a Nash equilibrium where they can't unilaterally slow down without essentially destroying their company.
So what? Anthropic believes their work has a 10% chance of killing all humans. I think risking the destruction of Anthropic's business should be worth avoiding that, if that's what they believe. And with one half of the frontier duopoly gone, the other half would have no incentive to race forward. And I know there's China, but they just get all their capabilities from distilling Claude, right? So, problem solved there, too.
Sure, but does slowing down the development of new models count as "adopting safety standards"? I very much doubt it.
> Anthropic believes their work has a 10% chance of killing all humans
This ignores the other part of what they believe, which is that they are the people most likely to make a model that doesn't do that. So, in their view, letting other companies win would increase the probability of human extinction.
> with one half of the frontier duopoly gone, the other half would have no incentive to race forward
I don't see how this could possibly be true, with at least half a dozen companies being just months behind what the frontier labs are releasing.
Of course! What slows down the development is the adoption of specific safety conditions the companies draw up. They can just do that, and it will hold up in court.
> in their view, letting other companies win would increase the probability of human extinction
Yes, I've heard: "We must be in charge even if we end up killing everyone in the process." I personally think that proposition is invalid, but we're all entitled to our opinions.
People used to append IANAL to such statements :-)
> I personally think that proposition is invalid
I agree, but that's meaningless in this context. I was responding to the claim that "if they wanted to pace the frontier, would simply do it", which is false, given what they actually believe.
Do you think their beliefs deserve some sort of special treatment?
No.
I mean, the point (if the claim is to be believed) isn't just to "slow down the development", it's to take more time during development to properly assess the risks the models pose, develop methodologies to reduce that risk, and standardize that across companies. I doubt those wouldn't count, especially in the eyes of regulators of an administration calling for that same slow down.
I hope this is a joke. But most of the LLM research and inventions come from China? The best papers are from DeepSeek? You either get the data by stealing from humans or distilling from bigger models?
Their fear-mongering about GLM 5.3 got me to try it out. Its very good, I'll only go back to Claude if GLM isn't available (it forgot how to do tool calls yesterday).
Interestingly enough, it seems to compact at about 10% of the 1mn context, which makes sense if they're trying to run profitably.
I recommend trying Coralbricks with GLM due to their cheap input cache prices. GLM can get quite expensive elsewhere.
On the last OpenAI release, they only had comparison with Anthropic models, on the last Anthropic release, they only had comparison with the OpenAI models, there's something obvious going on here.
Sources:
You don't compare yourself to the underdogs. Coke never made a "we're better than Pepsi" ad, but Pepsi definitely compared itself to Coke.
For example, Anthropic putting K3 in its comparisons would be a huge admission that K3 is worth considering.
> For example, Anthropic putting K3 in its comparisons would be a huge admission that K3 is worth considering.
They do put OpenAI though, if it would be a comparison only with their own models, why not I get it but a single other competitor?
It would be like Apple making comparison page with their new iPhone and only mentioning Samsung and nobody else for example, it would sound weird. You either include competitors or you do not and include none of them.
It does not matter if competitors would not also agree to destroy their companies.
If I was competing with a bunch of people on building something that I came to believe would be an extinction level event, I wouldn't be saying "Even if I slowed/stopped, the others would not, so I have to keep going". I'd say that I want absolutely nothing to do with pushing it further, stop my work, then regardless of consequences, do everything possible to stop my competitors regardless of legality
Is enforced by the federal government if they want to.
The most recently truly significant enforcement action was in the 80s, the AT&T breakup. And the current administration certainly will never enforce anything that hinders the oligarchs.
What’s that now? Surely you’re joking.
But that’s not the problem anyway.
If all the US AI players agree to self-regulate, that doesn’t help anyone. It only hurts the US / West.
The point is to get the government onboard so it can advocate for a global agreement.
Antitrust is a joke since the last decade. If we are going to not apply it, maybe we can get something positive out of it for once.
What a small price to pay for the survival of humanity.
If the danger they are claiming is really there, Amodei and Altman should already be serving in prison for taking destructive actions.
But no, all they want is a regulation against their competitors. So typical for Misanthropic and ClosedAI and thei paid shills.
I mean, these companies are saying AI will destroy humanity if it’s not paced. If they truly believe that, the small risk of destroying their own companies seems like a small price to pay.