The expected next step then would be for such judges to use AI to read the documents.

You then get a decision that your AI agent explains to you.

What could possibly go wrong? You will have to ask your agent for that.

It's always been a silly argument that's also predicated on a massive fallacy which everybody is aware of but seem to conveniently ignore during these arguments: that humans never make mistakes.

On top of that, dealing with text is literally what LLMs are all about.

The problem with accuracy, hallucinations and just generally ensuring responses are grounded in factual stuff is in large part and engineering problem.