Yeah, it's like declaring self driving safe because people are told to remain alert with their hands on the wheel, ready to take over in an instant. It's a charade.

That’s why most people that says LLM doesn’t work. It’s not that it can’t produce a good output once in a while, it’s that you can’t guarantee it. Or reduce the risks of a bad output. It’s a chaotic element and the cost of being alert enough to ensure consistency (if it’s feasible at all) is higher than just doing without.

But AI proponents are more than happier to brandish carefully curated anecdotes than to do a systematic study of risks and impacts.