In the "our biggest danger is a daily danger" camp, I prefer analogies like:

* Asbestos: AI models making business-decisions which cause a ton of unrecognized damage, which cannot be healed, and replacing them with reliable processes involves expensive re-architecting.

* Radium Toothpaste: AI assistants that end up dealing psychological damage, especially if they are pitched as your friend or confidante.

* Chemical spills and plant accidents: Oops, our server-banks were manufacturing our latest desperate attempt to Create Digital God and in the process we accidentally destroyed/crippled/compromises some people's sites and databases because we weren't really paying attention to what was going on.

I see parallels between this race and Mutually Assured Destruction (MAD).

I think we are in a MAD crisis, as the motivation to continue unsatisfactory control over development of AI is from unlimited capital inflow.

This is because we are looking at the trajectory not just their current capabilities. LLMs went from barely being unable to focus on a task to purposefully escaping sandboxes within like 2, maybe 3 years?