Consistently catching errors is vastly harder when you’re not the one doing the legwork. Send the wrong file, do a bad copy and paste, or skip one verification step, etc is all it takes for most people to majorly mess up.

For most professionals it’s just safer to avoid having output in your workflow. Keep using AI, but stop copying and pasting AI output.

It's not hard, but having to check work so carefully, at some point you're better off just doing it yourself.

Yes exactly. It isn’t a surprise that the industry with no professional standards worth discussing is also the one having the biggest field day with ML. My point is that in many other professional arenas acting like a software “engineer” is called malpractice.

Well hang on, "just work" implies an objective standard of correct vs incorrect. No such standard exists. I'm sure that there will be legal AIs that reliably do... something... but the whole point of the legal system is to allow space for humans to argue their case. If an AI "does legal" deterministically, without human intervention, how can it be said to be "working" if its very use defeats the purpose of the system it has learned to navigate so expertly?

That’s utopian thinking unburdened by reality or the need to explain how this will work. I have no interest in that.

We aren’t talking about the new version of SV spiritualism here, we’re talking about systems people actually care about.