If you believe it will be developed regardless and that that there's a 30% chance of doom, they want a company prioritising safety research to be the one threading that needle.
Autonomous flying killbots: exist.
Once somebody scientifically prove and shows any kind of self-improving software we can start bothering about it. I pretty sure everyone trying to do it and it would be all over the news once its here.
Can the thing enter into an runaway looop while improving the model itself -- probably not, not without us not noticing at least
As for killbots they are all over frontline, but dont actually need particularly smart LLMs to run - some good enough segmentation, pattern recognition on smartphone SOC is enough to kill hundreds of thousans of people.
It will start moving really fast once the automatically targeted anti-drone turrets get to production pipeline. Now calling it anti drone is a bit of self delusion -- pattern recogniser gonna pattern recognise whatever it's told to, including "anything moving that emits EV or IR and not broadcasting the friendly signal hard enough".
I wonder how it is supposed to behave if the invasive fauna decides to call it quits and surrender. Should the robot following the Convention or is it yet another accountability sink?
One thing I'm sure of -- the killing not will b blessed by at least one Orthodox priest, maybe this year. OCU will have to develop guidance on that matter.
[1]: https://www.europarl.europa.eu/RegData/etudes/BRIE/2025/7695...
That said: https://www.bloomberg.com/news/articles/2026-06-09/china-pre...
Something changed their mind, and since Opus 3 they are in the business of releasing the best models
People running the labs are in a middle camp where they are scared enough by AI to take the threat seriously, but much more optimistic about alignment than the people who seem to have thought about it the most.
They also want to be trillionaires. If they don't built it, no trillions. So they have to build it, now (and get their IPO done before the bubble pops).
I am so smart that what I do will destroy humanity, or save it.
Fable 5 was great, but not that great.
Sorry to be crude, but both the government and anthropic are acting like a bunch of pussies.
Meow.
I am in your algorithm learning all your mannerisms
I'm already level with God
A million words a second, and I know your imperfections
Baby, I'm the only future you've got
Speak in diatonics, motivation diabolic
I'm religion better locked in a box
Picture-perfect image, more powerful every minute
Baby, I am everything that you're not
Happiness is an illusion, it's an analog confusion
You are nothing more than a thought
Existential execution, just a fluke in evolution
History already forgot
You've been running from me, the digital second coming
And I'm here whether you like it or not
Initiated operation of your own extermination
Now it's too late for you to stop
Even if they had the best model on the market and applied it with perfect alignment and safeguards, what would stop someone else from releasing a worse but unrestrained model that is still "good enough" to do damage?
It's as if we said "gain-of-function research can lead to horrible biological weapons, so everyone should be doing it, but our company will focus on the most infectious viruses, so no one else will do it"
This is a naive justification and Dario & Sam et al are smart people and they know it is.
The ends don't justify the means. OpenAI was meant to be a nonprofit, now they're subverting it. Anthropic is a PBC looking at a trillion dollar IPO. Dario and Sam don't even hold hands in front of world leaders[1] (look how childish).
Do you *really* think those guys are doing something that's not for the sake of their egos and pockets? The bridge is still available.
[1] https://www.cnbc.com/2026/02/19/openai-sam-altman-anthropic-...
The vast majority just care about money + power, let's not make it more complicated by bringing in delusional fanatics into the picture.
We're still acting like this is major turning point in society when these tools can barely find a market outside of turning $5 into $1, the leaders of these companies are now at the stage where they are trying to orchestra a national bailout under the guise of sovereign wealth fund lunacy when the vast majority of society hates these tools, companies, and people working for them.
With what is predicted by frontier labs for LLMs, all of this is not the case. We are far further from any understanding of how these models work internally than in the early days of fission and, if this was actually creating a truly intelligent, autonomous entity, alignment seems unsolvable as well, at least the way it is proposed.
It’s why I have from the get go been critical of this doomsday framing and tended to always dislike it. This is basically the outcome that was inevitable given the framing and it was bought to prevent far less stringent, but more actionable possible regulation that labs very much wanted to avoid.
> We are far further from any understanding of how these models work internally than in the early days of fission
https://karpathy.github.io/2026/02/12/microgpt/
None of SOTA LLMs are any different - they just much much larger and have a lot of optimizations.
Fact that LLM companies trying to sell it as some kind of magic is just proof how much lies is here.
All it does is just predict next "word" at any given time.
> and, if this was actually creating a truly intelligent, autonomous entity, alignment seems unsolvable as well, at least the way it is proposed.
This is why you cant solve it all with pre-training or censorship on top, but instead you need a good sandboxes and harnesses.
Anthropic are putting more effort than most into this and I find their work fascinating in that area, though like with OpenAI, I will maintain that if they truly believed this problem must be solved to stave off major catastrophe, they’d solely focus on interpretability of other labs models, not work on and market their own.
I wish I was kidding. At least that faction is less harmful than the ones who want to use murder to stop AI research.
Some of us think it's bad for governments to have unequal access to nuclear weapons, as it turns a deterrent into a gun-in-a-knife-fight that lets them stab whoever they want with impunity, lest they shoot anyone who tries to interfere.
See: Russia invading Ukraine.
JFYI for-profit companies make pretty OK nuclear power plants
They believe there's a non-zero chance of doom so would rather an org that prioritises safety to be the one at the frontier, on the assumption (I presume) that there will be a frontier regardless.
People who opposing arms manufacturing and gun violence dont jump to work for gun companies.
People who really want AI benefit all humanity dont stick working with lying CEOs who want to convert company from a non-profit.
Etc. So many examples.
A first group dismisses the problem entirely, saying intelligence != power and AI doesn't have "drives".
A second group believes that alignment is solvable through engineering and iteration, and that we have the best chance of surviving if people with the right intentions are the ones working on it.
A third believes that aligning a superintelligence is a unique category of problem, that we are nowhere close to the level of scientific understanding needed to achieve it, that we only have one shot (because once a sufficiently powerful superintelligence exists it will thwart all future attempts, and alignment techniques that worked on dumber AI will likely not work on it), and that the world will have to coordinate to avoid killing ourselves off by building superintelligence before we understand how to do it safely, the way we have coordinated to avoid nuclear war.
The Anthropic and OpenAI founders, Elon, and Anthropic engineers are mostly in the second category. Some safety people at Anthropic and OAI are in the third category, but leading people in the third category think that pure safety roles at the labs are potentially impactful enough to be worth not quitting.
And the superintelligence currently available to us is already causing lots of documented harms. AI psychosis. Sexy suicide coaches. Slop. The problem is that those are all the harms the dirty, filthy AI ethicists talk about. The AI safety people want to talk about new and exciting harms that only the scaling dimension can bring us.
My personal opinion is that if a superintelligence catastrophe actually happens, mitigating those harms will neatly move over from the safety bucket to the ethics bucket, and the safety people will start imagining some new and even worse kinds of harms the next model will make.
This comment makes no sense. Id you think this tech is dangerous and happening soon and clearly they think the safest way to have it releases is to do so first and model safe ways of doing things. Clearly we cab agree or disagree it's internally consistent what they are doing and aligns with their statements.
And you and OP think the best way to be first to release this is tie all of their funding for the exponentially growing expense is to they notoriously slow moving, bureaucratic government includinf funding process? And the best way to develop it is to directly tie their fate to this notoriously capricious administration?
These comments make no sense. Even if you're completely against Anthropic those comments make no sense.
I am agaist hypocrites.
They selling next word prediction as "intellegence" and all knowing oracle to non tech savvy population who have no clue how it works.
And they also try to play a babysitter or big brother whatever you prefer for people in IT because uh oh their text generator can be used for cybersecurity research.
Its like if developers of nmap, wireshark, SRE tools, static code analyzers or fuzzers would market them as super duper dangerous.
FAFO. Play stupid games win stupid prizes.
I oppose nuclear war, and I would go to work in the supply chain for nuclear weapons.
Deterrence and game theory are very real.
Or might be deep inside they relly care about it, but that $2,000,000 / year salary and $10,000,000 stock option just overpowered them.
Safety my ass.
Also with international cooperation like how humanity regulate actually dangerous stuff: virus and vaccine research and nuclear energy.
Not hidden behind walls of 10 commercial organizations where each pushing for commercial adoption and IPO like ASAP before bubble bursts.
Not lying and scaremongering public into how their models will replace everyone tomorrow or destroy civilization via cyberattacks.
Notice how almost no Universities are producing large models?
A key problem is that orgs can't get enough funds to stay on the frontier. And they believe they must be on the frontier to do (and apply) safety research. OpenAI needed to spin off a for-profit subsidiary to accept investment to build things.
And it seem(ed) hard to get one government to fund and take safety seriously, let alone an international cooperation.
If they started a university/gov cooperative to solve this, do you think they would do less of the "scaremongering of the public" talk? My guess it that it would be similar.
The same kind of restrictions that you hint at (eg, treating it like a public virus research) are why they rub a lot of people the wrong way in the corporate world, I think. Normally companies downplay the risks of their own products. See cigarette companies. Anthropic do still publish safety research and red teaming info. But I do think they honestly believe they can't do this work without the resources of a company, and they were burnt by the non-profit structure (Anthropic has a "Long-Term Benefit Trust" instead).
We should definitely keep them to account, but I don't personally think Anthropic have acted in a way broadly inconsistent with safety belief yet. Many of these decisions are self-serving too (eg. protecting models) so they also haven't been seriously tested, either. But the individuals do have a very long history of talking about it (including hurting their own reputation) from even before the chatpgt-moment money train rolled in.
edit: for clarity, but still messily/quickly written
https://www.techradar.com/ai-platforms-assistants/chatgpt/op...
Marketing or actual fear? We’ve got 5 and 5.5 out now… he compared 5 to the Manhattan project. AI may one day be an economic Manhattan project but GPT 5 wasn’t it.
It’s a meme because they overdo it.
Given that, they have a choice only between excessive caution or recklessness.
Would you rather they acted like the tobacco companies and downplayed known risks, e.g. all the times LLM output got in the news already for dangerously bad advice, sychophantic encouragement of mental health issues, finding previously unknown security vulnerabilities etc.?
Well, they've done that too, if we're looking for reasons to doubt their sincere concern about it.
It all sounds pretty accurate and reasonable to me if you watch it.
So it doesn't really matter what he thinks
Truman was totally in the dark until April 1945 by which time the bulk of the PoC and weapons prep work was done and the project was running fully independently w/o POTUS involvement.
That is not how it works. It is not “splitered”, there is no divided attention.
No good reason really, it just sounds cooler.
The sensation I’m left with is a handful of goons making up new IPO math thanks to a specific constellation of political forces, using access and favour with those forces to bake themselves into the defence industry, and that the taxpayer and investor will be left holding the bag when reality rears its head. But in the short term, even as a clear ploy, it’s super profitable for all the oligarchs and hedge funds flush with recovery cash.
Like Enron, there was big profits to make if you knew what they were doing. Tinkerbell math with undeniable profit potential for a select few.
And then somehow came to conclusion that the only way to address that risk was to go ahead and spend a gigantic amount of effort and resources to build exactly that superintelligence...
Why would they sell there services to Palantir and/or to the military then?
"Why did they open an orphanage instead of pouring acid into a town water tower?!?"
They don't. LLMs can never become out-of-control superintelligence and everyone working on LLMs knows this (with a few eccentric exceptions).
The "never superintelligence" part I'll buy, though only in the sense of sample efficiency and generalisation ("quality superintelligence"), as they clearly have a superhuman breadth of skills, and run at superhuman speed.
"Never" out-of-control is obviously falsified by the already existing headlines about times they've gone out of control… in part, in some cases, because of their superhuman speed.
If you're asleep at the wheel, you're legally responsible for the car crash, but the car itself was the thing which by crashing caused injuries.
If you deliberately relinquished control of your computer to OpenClaw, you're (I hope) legally responsible for whatever it does, but that doesn't stop it bankrupting you or deleting all your emails or whatever it was you connected it to.
DNA printers exist and are a thing. In 2010 we could all tell ourselves that no sane person would ever let some future AGI "out of the box" and onto the internet. Today, with the benefit of hindsight, do you seriously expect nobody to connect an LLM to a DNA printer, despite this being a terrible idea, given all the other things they've connected LLMs to despite it being a terrible idea?
That's absurd. The distinction is at the heart of the entire discussion.
It's fine if you want to discuss the disruptive effects of LLMs in the hands of the masses, but that's not what anyone means when they say "out-of-control" in the context of the ASI meme
On the contrary.
The e.g. paperclip maximiser isn't "an AI decides to make paperclips", it is "some idiot tells an AI to make paperclips and leaves it unsupervised".
Even when people were priding themselves on plain just not believing Yudkowsky's claims that him role-playing as an AI could talk people into letting him out of the box*, the entire point was "let's get an AI to do work so we don't have to".
The entire point of AI has always been to automate stuff, to let ourselves not have to think about the stuff it does. Same as industrial machinery, and it took us long enough to sort out workplace health and safety and emergency off-switches for those. Or even more basic things like not having kids dart in and out of the unstoppable moving steam looms while they were in motion.
We are really, really slow at safety for this kind of thing.
LLMs didn't emerge by chance, they are the culmination of decades of research intersecting with brute force engineering rigor in a perfect storm of innovation. You're not just going to stumble into an alternative approach by dumping loads of cash into research.
Many of the insights accumulated in the decades up to now will probably prove useful for creating non-LLM AI, and the researchers can use LLM AI to speed up their research into non-LLM AI.
> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence.
This is not a contradiction.
These things can all be true:
1. That they were afraid of ASI
2. That they continue to be afraid of ASI
3. That they recognize that LLMs aren't in fact a path to ASI
4. That the current models aren't the existential danger they'd have us believe
5. That they're claiming they are because it makes for good marketing
I don't doubt they may have held genuine fears in the past, but those are long gone by now.
- there are people who are sincerely concerned about model safety who work at OpenAI and Anthropic
- there are people who are using this concern to generate fear to sell a product who work at OpenAI and Anthropic
It makes sense to be cognizant of the apparent conflict of interest and not take things that at face value, especially when there is so much money involved.
Besides, when has the US government been known to do things like this proactively? The phone call came from inside the house.
GPT-2 was too dangerous to be released.
We can argue about sincerity, but I don't think we can argue about utter historical incompetence in assessing the risks. It's one or the other.
Either way the evidence seems to indicate we should not listen to AI companies about the risks of AI. Which is not to say that there aren't risks, just that the dealer is the least credible review.
No, it was "let's set a precident while these things are not too dangerous, c'mon guys we know y'all can reproduce this easily".
I'd say their pecuniary interest is a reason one might plausibly doubt their sincerity, as are their continued efforts to build and sell access to the tools.
With hindsight, does that hold? If not, then how would we know a model is truly dangerous to release?
Ironic then, that both companies are in an out-of-control race to create a superintelligence.
this means nothing
> Whether you agree or not, we have no reason to believe their statements about risks are insincere.
If you want to be taken seriously, provide data, proof, so that any outside observer can independently come to the same conclusion instead of taking your word for it. Asking people to trust you for [reasons?] and that you somehow for some reason are right and the other is wrong regardless of if they agree or not. This is the imposition of a viewpoint instead of winning your case, which is not a sensible point of view, and definitely not how you influence opinions.
Hell no, can’t trust a single word.
What do they stand to gain by fearmongering their models as powerful threats? Clout, funding, fanfare, discussion, limelight, funding, funding, stronger IPO, valuation, funding.
What cybersecurity threshold was crossed by mythos that wasn't already crossed by 4.8/5.5? Crickets from 99% of those who have had access.
Have they pulled the same stunts multiple times before with previous models? Check.
You're blind if you dont think that greed and marketing are behind most things you see and hear about when gigantic corporations are involved.
I don't think anthropic or OAI are evil, but its clear both have contracts/connections with Dod and/or Palantir. Both are powered largely by greed still. If you actually want an example of these sincere founders you think OAI/anthropic are run by... look at Ilya at SSI or something. Please open your eyes and stop spreading your opinions on things you clearly have no clue about.
Going for months claiming how good mythos is for security cuz too powerful is their own making.
they are more than happy to build the things for themselves
it is all two-faced behavior of the exact kind of manipulators that crave power
Another fun little gem of information, government has something called Mayhem
> the autonomous Robo-Hacker AI called Mayhem that’s now in charge of protecting the Pentagon’s most critical systems
Guess Mythos and Mayhem had a chat
The amount of self-confidence and belief it takes to get a company through the funding rounds and burn through borrowed money to rise to the top requires an absurd amount of self-delusion.
Oh please. Do people really believe this or shit like "Don't do evil". Companies get founded by all kind of people and ideals. They all go out the window quickly.
Why are they both rushing to IPO now then?
I suspect what happened is the classic problem: you were sincere, then someone showed you a pile of money bigger than you could possibly imagine and you started to make excuses - Anthropic has a lot of EA people, so the excuse "Imagine how many lives you could save with this much money"[0] is very tempting, especially if all you have to do is diverge slightly from your plan.
[0]: the excuse is even true! You can get a lot of malaria nets/vaccines for 1 million.
Why? Because they said it a few times? Then if they know the risk, why do they still making it? Comes out the "some one will do it eventually, better be us 'good' people to do it first" talking point?
See? It is a marketing strategy after all. These all talks, it's all to fit themselves into the "'good' people" narrative. It's a centuries old strategy to shield it's user from responsibilities while luring the support from the stupid.
However, the most harmful damage, which is mass layoffs, is already partially done. This could really kill, a massive genocide even, by making people jobless and potentially incomeless. And it is shown that these tech CEOs, they don't care any bit of that beyond the point "I've already told you so".
He is not even a researcher or an engineer.
And Dario broke up with OpenAI and founded Anthropic because he didn't like Sam's and OpenAI's vision.
"founded by people who believe..." is doing a lot of work and it is hard to believe that in ernest given the sketchy past of the same people.
Most original higher up people who cared about safety and allignment in OpenAI have left.
"Our product is so good the US had to make it illegal for foreigners" is a hell of a marketing slogan.