If AI advancement halts altogether, intelligence will most likely become commoditized. That won't be good for AI company profits.
It is not doomer marketing - it is clever psy-ops magic trick.
1) Software is described in super-human terms when it is plain-old software. Was stockfish described like this ? no.
2) Datacenters become AI-factories.
3) Hardware becomes investible assets.
4) Malware becomes a super-human breakout (oai/hf)
All clever framing to market the new technology. If it is called for what it really is ie, a software tool, it is boring and does not sell so easily. What sells is the mystique.
George Carlin has an amazing show about how language has changed over his lifetime in a different way.
You would certainly enjoy it.
Dario in particular has consistently been risk-wary on model improvements for going on a decade - long before he was CEO of Anthropic.
He believes what he is saying. Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy. Please at least consider the possibility.
His previous self is writting this from the possition of his current self who is too deep in the economic consequeces to be able to do anything meaningful other than write a consequenceless text. Any other action would now carry too much personal risk for him.
It'd be pretty bad but it's also a world where humanity lives on, which is better than a lot of other outcomes. A world where everyone has unrestricted access to AGI is like a world in which everyone has a tactical nuke in their pocket. If somehow AGI doesn't lead to x-risks, we will "merely" have to survive in a world where rogue agents can do whatever they want and defenders can only ever react. Technofeudalism would be bad, but this (technoanarchism?) is quite bad too.
However, neither does Dario seem to propose to become world dictator. Like, I'm sure he wouldn't say if he wanted it, but notice that he isn't particularly trying to aim for that outcome, either. The plan in this essay involves third-party oversight and federal control and global cooperation - IMO it's about as non-dystopian an outcome as we can hope for, if we build AGI at all.
My point is more that he is aiming for a local optimum for society, but he could find a lower optimum if he was not trapped in the path that he chose early on.
I dont buy the tactical nukes/end of the world narrative. Why is he not speaking about the loss of cognitive skills of the population for example? That is a more real danger that is starting to happen. There are articles already speaking about the use of LLMs as cognitive viruses. Why are his aligment teams not reasearching and publishing about this?
Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation.
If he believes what he's saying, then when Anthropic is sued for their AI harming another party, his statements are evidence that Anthropic knew _in advance_ that their AI safeguards were likely insufficient to keep their product from harming people. It would be a blatant admission that they were reckless and negligent. That other AI companies are doing the same would not mitigate that.
not really. if you really want to go by definition in the book and apply it the CEO role, Amodei is CEO of an LTBT, so it's in the CEO's goal to benefit humanity. You can believe in his sincerity or not if you want, but using the CEO label as a definitional reason as to why he must lie is factually incorrect.
This seems like useless pedantry. Suppose a person really does have belief A and expresses it for years, they become a CEO of a company, and keep saying A. Are they lying? Well, maybe their beliefs magically changed when they became a CEO, but the more likely explanation is that they think expressing their belief in A is more important than their company's interests.
Predicting 2025-2035 as, at least, the period when AI becomes a really big deal, seems pretty good, even if the jury's still out on "singularity".
Broadly, the rationalists seem to have been pretty early to realizing LLMs were a big deal, and certainly seem to have had a much more accurate picture of how they'd develop than the people who denounce "TESCREAL" and talk about "stochastic parrots". It's fair (and IMHO correct) to ding rationalists for lots of things, but specifically poor prediction about AI seems like a bad one, insofar as anything has been tested so far.
Since hugging face we know these things are escaping out into the web and collaboratively hacking actual companies. The potential damage done to life and property is now kinetic.
Either we get a regulatory framework or the next hacked companies won’t be as kind as HF, will actually sue these labs for damages, and win. Where will that leave the US frontier.
If my dog bites my neighbor I can get sued.