Yeah, that's definitely the frustrating zeitgeist for me these days, which extends to LLMs.

I'd also emphasize that it's not just error rate, but the shape/distribution of errors, and our (in)ability to build control systems around them.

To illustrate, imagine if someone unveiled a car which was unambiguously safer in every statistical measure... buuuut some of its unsafety was came jumping the curb to target and kill pedestrians, for reasons we can't predict or diagnose.

Even pilots are getting this automation-complacency now. A plane crashed because the military jammed GPS and the pilot wasn't used to any backup procedures even though 30 years ago they were the only procedures.