I agree completely you always need to check the work of LLM agents, but it does strike me as a tiny bit funny to anthropomorphize AI by using ‘trust’ while warning against anthropomorphizing the AI by using unchecked output. ;) Generally speaking, “trust” in AI has been going up very quickly as the models & harnesses improve, and as people figure out effective workflows.
I trust my hammer with nails but not screws… does that mean the hammer should generally not be trusted? The problem with AI is we don’t know the difference between nails and screws. (This may be where my analogy breaks down. :P) But I feel like saying don’t trust it isn’t as helpful as saying something like you should expect to spend more time planning and iterating than before, and you should expect tot spend more time reviewing and checking output than before, and learn how to use skills and context and subagents, and learn to use AI on some non-production low-consequence projects first. Saying ‘generally not to be trusted’ implicitly suggests not using AI, and doesn’t leave the reader with how to use AI. The goal is to build trust by building good workflows and by understanding what works well and what doesn’t, right?
I trust a hammer to be able to hit a nail, without breaking. But if the hammer is old and the wood brittle, I don't trust it anymore.
Using it for anything else (screws) has nothing to do with trust, but using the wrong tool.
- nice to have but clearly not important enough to invest time in
- definitely not worthwhile paying someone to do
- not going to hurt anything or anyone if it wasn’t implemented as close to what was envisioned.
If you read between the lines then trust may also mean privacy, if you’re furthering the goal of some company that may be stealing your data for training even when they say they are not because there are legal loopholes that allows them to get away with it, etc.
Your example of hiring Donald Knuth to write your code doesn’t fit into what’s being said about trust either. If you were never going to trust anyone to write your code, it doesn’t matter who it is anyway.
For most of us, even if we are engineers, chances are hiring someone legendarily good at writing code to requirement will produce far better results than what we knew we could achieve.
I would trust someone who is that good to write the code — more than myself — to do things I want to do it better than I can while being able to catch all the things that didn’t realize I needed.
There are many AI bulls who adamantly disagree and cite Tao’s statements about LLMs for mathematical proofs as an example of how advanced and autonomous these systems already are