A savant who confabulates about 10 % of the time, per the best benchmarks, no matter how much “thinking” it does (GPT-based LLMs will always “hallucinate”)? How do you trust it?
replyI would rather take the time to ask the same question to the Emacs contributors mailing-list, who have the incentive to check their posts for mistakes.
Have you ever worked with a modern agent? They let many weak assumptions slip because their cognitive functions aren't remotely as deep as human's (that's what makes their code a horror), but they have also been trained to hyperfocus on verifying and grounding everything they do and say, you have to actively stop them from doing so when it's unnecessary. This not what I can call confabulation in any good faith, let alone 10% of the time. I think you've been misled by the old paper you cite below, it doesn't say what you likely think it says.
replyYes, I mainly work with OpenCode, with GLM as an orchestrator agent and DeepSeek Flash as a worker agent. I’m assuming it’s modern? I don’t have the money to use either Fable or Astra, but my assumption is that the few more points they get on the benchmarks still don’t make up for the wild confabulated things they do in my projects.
replyBoth are fine, though frontier models are more persistent. They mostly take unwarranted shortcuts, trying to find the easiest way to reach the literal goal set by you, failing to see the broader picture. But they definitely do not confabulate, as in they don't just make up nonexistent things, they would rather walk extra mile to verify when they're not certain, which is the reason the software they make is trash but it works in the narrow sense. They can be rigid in their writing style due to mode collapse, but I haven't seen this leading to them inventing nonexistent functions or anything like that in a long while (and it's a job for formal tools to catch that, in any case).
replyModern LLMs are very far from being next token predictors trained only on text. Text and other modalities is the base that makes them understand your language and have concepts similar to yours, but they were also subjected to RL that makes them good at reaching the goals. Two tricks in a trenchcoat instead of just one.
> But they definitely do not confabulate, as in they don't just make up nonexistent things
replyI've definitely seen both Labs' "best" stuff confabulate to this day.
I can't speak for the other providers but Gemini carbon feels like the difference between gpt3 and gpt5 when compared to Chinese open models.
replyHow do you trust a human? Are humans infallible, or at least better than AI at explaining things? I known my answer.
replyI trust the Emacs contributors, the actual experts in the topic, to fact-check their own answers, because they are incentivized for doing so (reputation). They are also able to tell when they aren’t sure about a particular topic.
replyGPT-based LLMs, by design, cannot. https://arxiv.org/abs/2409.05746