upvote
No, they are two very distinct groups of people who have one commonality, that of talking about rogue AIs. It is as if you are unable to distinguish Dr Waldman from Dr Frankenstein. (https://en.wikipedia.org/wiki/Doctor_Waldman)
reply
Thankyou for the correction, and for continuing the analogy. Unfortunately when the peasants get their pitchforks and torches, they may not distinguish the Dr Waldmans from the Dr Frankenteins either. Hopefully they will.

Anyway, that was not really the point I was trying to get over. These systems that OpenAI and Anthropic and so on are making are not individual AI ('corpses') that have gone out of alignment ('spontaneously revived') and gone wild. They are swarms ('stiched together') and were prompted to do exactly things like this ('struck by lightning'). Ok enough with that analogy, it's dead.

The larger point is that it is unconvincing of these companies to claim that these systems were 'out of control' when they effectively set up a complex system, in the technical sense of a large number of entities with diverse interactions between them. Emergent or surprising behaviour was bound to happen. Then, finally, they prompted it with the equivalent of "hack the world, make no mistakes" then were shocked, shocked that it used all sorts of unexpected tricks to do so.

reply
Ah, I see I was confused - I was thinking of what are now called “AI doomers”, although when I knew them they were called “rationalists”. Everything OpenAI and Anthropic say about rogue AIs, even the terms “alignment”, “AI safety”, “AGI”, they are all cribbed wholesale from what these people were worrying and writing about over the prior two decades. But for most people, they have only heard CEOs of AI companies say this kind of stuff, so that’s who they are thinking of.

I agree completely that the companies are complicit and should have expected exactly this to happen.

reply
I am not seeing MIRI prioritizing capabilities over safety/alignment research.
reply
You mean people like Yudkowsky?
reply