And even if they are related - if Opus 4.8 always has a 1:100 chance of a specific hallucination - then running the same model twice does indeed dramatically reduce the odds of an error in the final output.
Yes, LLMs can be SOTA for NLP, but you’re going to have to use them to write software or workflows that are more deterministic.
Personally I think this is a bad characterization of using LLMs to fix up LLMs because while you can never guarantee results this way (as the quoted line claims here, which is worthy of criticism), it is, in practice, useful to use LLMs on top of LLMs. And there's no infinite regress. Auto-mode in Claude Code, for example, seems to me like it's been successful at making the system more safe than --dangerously-bypass-permissions without prompting the user for permissions constantly.
What triggered my response was the “just review the output with another LLM and it’s perfectly correct”
Full story in the book
People don’t even have to be lying to be wrong about this stuff. Someone can learn enough about a topic to be halfway up Mt. Stupid in no time flat, and in doing so, think they not only truly understand the topic at hand, but might be particularly adept because they were such quick studies. People that know less are impressed, because why wouldn’t they be? Anybody that knows more than them sounds like an expert. And people that know what they’re talking about cringe at the overconfidence, and probably try not to engage: who wants to have to prove that someone’s boundless confidence is entirely baseless? Most of the time, they think the actual expert is full of shit because they think they’re the expert. It’s incredible how many times I’ve had people in tech confidently, even smugly “explain” design concepts and strategies to me that they did not actually understand, knowing I was an experienced, degree-holding designer… and they didn’t even have a chatbot’s lips on their ass telling them how smart and insightful they were.
Opus 5 first built me a detailed plan, but a couple important details were either obviously wrong or felt unnecessary. I went back and forth asking for sources and more information probably like 4 times and every time it did the "in looking at things in more detail it appears my previous advice was incorrect" spiel. It just became exhausting at some point because it feels like it really lays bare how LLMs are just minimizing that loss function but don't actually "understand" anything. It was really useful as a search engine (it correlated some highly relevant source docs), but I just couldn't trust it to believe it was actually done at any step.
I know it sounds silly but 1 layer ends uo being way worse than 2.
i can't even get agents to remember core instructions like "use jq instead of writing a python script to parse some json"..
I'm researching causal inference right now, and my main goal was to make sure I understand how to test estimation on synthetic data.
Basically, it's the same way it works with people. If you delegate a task that you don't understand, and you can't have a credibility proof (i.e. doctors, lawyers), then you research a topic well enough to be able to (1) define the task and (2) verify the end result.