We are increasingly seeing "Sorry out <Black box> made a mistake, the developers are fixing it" rather than any one person taking responsibility for the decision. This is a growing problem that is made worse by AI. Over-inflated trust in AI output is also documented issue; "The perfect billion dollar targeting system with a 99.9% success-rate (on its verification dataset)" just made a "fluke"; "Nothing could have stopped this tragedy"

When it was just called a "database query" or "algorithmic estimation", it was more implied that the actual decision is left to be made and that no analysis had actually been done yet. But when AI preemptively says "I've analyzed every source of data and come to the conclusion that this ship is carrying nuclear bomb material"; it really sounds urgent doesn't it? It probably doubled down when asked about it.

https://edition.cnn.com/2026/09/18/politics/us-military-ai-f...