upvote
Or he believes that other labs might get there first, and he is working to counter that threat.

This is the Manhattan Project again.

reply
How exactly does that work? The nuclear system of MAD relies on physical threat, lab A achieving ASI (artificial scary intelligence) does not prevent lab B achieving it.

I would like everyone involved to be a lot clearer about their threat models, with plausible series of clearly linked steps, rather than just sounding like a Vernor Vinge novel.

reply
deleted
reply
Yeah, super ethical lab earnestly believing AI has 10% chance to kill all humans within a decade and doing utmost to protect humanity is partnering with MIC and Palantir in particular. Sounds about right.

Before you tell me about how the CEO has taken a principled stand: on record, he had no problem using it against 95%+ of humanity outside the U.S., and was only against fully automatic AI killing machines, and only citing the technical reality of then-current gen tech, so one should read that as human-rubber-stamped AI killing machines are totally fine with him.

reply
Or he is paid >1M USD per year
reply
If you believe that the probability of destruction is currently 10%, but the probability of destruction if you decide to quit Anthropic becomes (say) 13%, then the rational move (Assuming you are opposed to destruction) is not to quit.
reply
The real rationale fot them not to quit Anthropic is the 100% probability of losing in income.
reply
> But by not quitting it's clear that he himself doesn't really believe

I’m not saying I’d press a button that had a 50/50 chance of ending humanity vs giving me generational wealth, but … I can see how someone would get there? There are percentage odds and payouts to match all risk appetites and valuations.

reply
A simple explanation:

This is a highly uncertain and dangerous scenario. Breeding ground for anxiety.

High agency people often deal (cope) with anxiety by trying to control outcomes. Some just flee the situation altogether.

We have an example of both here: one employee leaves, one stays.

reply