Before we would find an intriguing post on the internet from years ago, and you have to verify it with additional research--it's easy to skip that additional research.
With a LLM when you're skeptical you can interogate it. One thing we know for sure is LLMs are quick to admit mistakes were made when interrogated, comically so. A LLM might not always recognize its own mistake, but at least it is available for easy interogation, unlike the forum posts of old.
Manual research from reputable sources remains an option.
In my experience every LLM out there is utterly useless and quickly defaults into "here are other concerts that took place around that time near that location". Google Search (ignoring the AI overview) is even more useless, as it refuses to show literally any webpage that's older than say 5 years. YouTube search is genuinely better than Google at surfacing old and grainy fan-made videos uploaded in like 2010, but also defaults into synonyms nonsense pretty quickly.
But, the search functionality of exactly one forum and three local news websites that I know have an archive that dates back long enough beats every single one of those abovementioned every single time. Three people are talking about their experience at a concert on a random 15+ year old forum thread? It happened. The tiny list of 5 or so (Google-hosted!) Blogspot blogs I have bookmarked? They usually have a photo of the ticket that Google Images refuses to show me.
Not only are search engines completely dead as a category, but LLMs are a shit replacement for them. "We" (okay, Google specifically) has truly committed a crime comparable to burning the Library of Alexandria. Everything older than a decade that wasn't properly documented on Wikipedia is just gone, never to be seen again.
The funny part of this is that Google search is intentionally bad at returning YouTube videos, presumably because some anti-trust action scared them into artificially ranking videos from local news sites, Facebook, and other ad-walled content ahead of YouTube videos. Seriously, go watch a YouTube video, then try googling its title with “video” appended to it, and see if the “Videos” tab of google search ranks it as the first result.
It's usually uploaded by a channel with like 20 subscribers and has maybe like 300 views, but YouTube would rather show me some artist playing a similar genre on the other side of the continent with millions of views that was recently uploaded than a video from an event I specifically typed into a search bar.
It's a great tool, but verify the important things (or do them yourself)
Signal to noise has taken a dramatic hit.
AI shouldn’t have any reason to lie. But its lies aren’t intentional. It’s just actually making things up and “hallucinating” when it pretends that an option or setting exists, or confidently claims something entirely untrue, and makes up a source to go with it. For something Google is willing to shove into the top of every search result it’s crazy the percentage of time the answer is blatantly incorrect.
The goal of LLM's, as they are marketed now, is to drive engagement and stickyness of products. A wrong answer is brushed off with a "Hey, you're right, let's try that again" - a response purposely designed to maximise the friendliness of the system and minimise the sting of a wrong answer. The fact that an LLM will not respond to the same question in the same way twice (i.e. the 'temperature' ) is because increased accuracy will not drive engagement and, therefore, increased accuracy cannot be allowed to get in the way of engagement.
Psychics are still in business. Although to be fair they probably don't have as much revenue. I think people really enjoy being told how smart and insightful they are, and how much they've really cut to the crux of the issue. This isn't the whole thing, but I think it counts for a lot.
Even today, going to a therapist and having 1:1 sessions is a rational activity, and even covered by insurance. What do we hope to derive from therapy sessions but some personal insight and improvement?
You know, I go to church, and from my perspective, sometimes the most difficult discernment for an individual is between "The Holy Spirit's message to us in general" or "the general messaging to everyone around us" vs. "what I receive and my personal interpretation of things".
When preachers and oracles and leaders are speaking in generalities and trying to get big followings and trying to appeal to the widest audiences, that's when it's most difficult for us to determine what God is really saying to us, in our own hearts; that special instruction for our own lives. We can't actually get that from an oracle, psychic, or any 3rd party. It really needs to come from our own well-formed conscience.
And it's the same with a chatbot or LLM conversation. We can pose questions, make prompts, and get spammed with tokens and walls of text. No matter how personalized, it's still up to us to interpret that, and extract nuggets of news that we can use. It always has been.
I've started testing identical prompts across models, just to clearly see the "Yes, ..." with the "..but.." buried.
Example: "Can you view the salvage car in the lot before it is scheduled for sale?" "Yes, cars are viewable on the lot by members before they go on sale, blah, blah, blah." Then later "However, cars not yet scheduled for sale are not allowed to be viewed at the lot".
Example: "Can I control the keyboard shortcut Chrome has hijacked so that my existing OS keyboard shortcut will work even while Chrome has focus?" "Yes, chrome keyboard shortcuts, blah, blah, blah," and later "No, chrome does not allow ... ".
There is clearly some recent implementation of the idea that starting with "Yes, ..." has some benefit, but I'm having trouble adapting to the feeling of being lied to as a policy.
We live in hope that people stop doing stupid things and are constantly disappointed.
The last sentence reflects a lot of my feelings on the first question. LLMs have a sort of weaponized take on the ELIZA Effect. The better their memory the better they are at playing to human social desires to be listened to in an active conversation. At some point it stops mattering if the answers are right when the answers feel right, but really, like ELIZA back in the day, so much of what makes it seem special is just reflecting your own writing back at you in a convincing and persuasive way.
Steve Yegge likened LLMs to slot machines. The human brain is very vulnerable to random reward systems. If you get an hallucination, just pull the lever once more.
I'm reminded of that person who killed themselves due to their discussion with ChatGPT and their parent wrote their obituary using ChatGPT. I don't think it is enough.
People still respond to ads and political speeches.
I feel like the most pragmatic perspective is "trust but verify."
This is why they're so effective at coding: you can run the code yourself (or the test suite) to verify that it actually does what it's supposed to.
And maybe these people finding hallucinated results on Rachel's site are doing verification too.
This perspective I really don't get.
What has any of the LLM companies done to earn my trust? I lean more towards "verify because I don't trust".
Not necessarily, namely because P != NP. Verifying the correctness of a solution is faster than solving it. Thus a system that outputs 99% incorrect solutions and 1% correct solutions can still be incredibly useful.
not saying anything new. easy enough to frame it like any other assistant and check references
I'll temper that slightly by saying it's mostly out of morbid curiosity because the things that the Dreaming Piracy Robot comes up with are frequently wildly incorrect code, but it's interesting to think about how it might have got there.
And then I think, well, maybe Special Needs Wintermute has a point. Maybe there's a different way to think about it that I've missed.
And then I just change it back to what I wanted in the first place.
It's painful to watch my older colleagues use their agents, and they're not even that much older. Like they were intentionally trying to sabotage themselves sometimes.
They're getting better, but the time it takes for them to pick things up is just significantly longer, not the least because they're kind of just throttling themselves in addition.
Good thing that there's not much to pick up on at least.
They ask self serving questions, underspecify their requests, omit crucial context that the agent is blatantly not going to have access to, or subtly misdirect the agent. They expect the agent to figure out everything: you'll never catch them write a prompt longer than one or two sentences. They never steer the agent or look at the CoT traces.
My boss being a particularly poor case: he apparently has the habit of arguing with the agent, as if it was a person, as if there was any merit to that. Starts being a dickhead with it, shouts at it, what have you. Was flabbergasted we don't.
On the more practical side, they have zero mental model of the harness they're using (Copilot Chat in VS Code). They're surprised when the cheap-ass Auto model, which is almost always some beyond-demented version of GPT, does stupid things. They have no concept of skills, zero understanding of what an MCP server is, haven't heard of lifecycle hooks, agent memory, the various fs scopes (session, workspace, user). No concept of how to have the agent inspect its own debug logs for higher accuracy action provenance.
This also snowballs. Having to give them a stock config is one thing, but even beyond that, you won't see them experimenting. The MCP you're using doesn't support some action? They'll never interrogate whether the underlying scoped OAuth token or bearer token does support it, and they'll never ask their agent to patch the functionality in. They'll not consider the various user flows it can perform on their behalf. They'll not string them together into end-to-end automated workflows unless you explain it to them this is possible, and even after that, they'll just kind of ignore it. They'll never build tooling, extend the harnessing, etc.
Whether this has more to do with age or just disinterest-induced lackluster adoption, up for opinion.
I've noticed the same effect, but the lines it always seems to fall on are if the person fails one (or heaven forbid both) of these: 1) are you curious about how your tools work and how to get better using them? 2) can you hold the mental map of both what you are solving and how your tools work in your head, and explain how information flows.
There is also a dash of: 3) are you willing to try something, even if it has a bit of a screwup risk, just to see what happens?
Why not just actually do the thing, instead?
I did give up on cheaper models, they required constant babysitting, and in those cases yes, the benefits indeed evaporated. The expensive models have been genuinely working wonders though, and were still able to justify themselves economically plenty, at least by my own measurements.
I think there's also one underappreciated and indirect way agents help with productivity: they counteract the attention span collapse of the past years. By being addictive themselves, they keep you engaged, and being engaged means being productive. Not even asking an agent to check something out feels too rich, and once you've asked, you're already one foot into the flow.
There's also definitely been some honeymoon effect going on for me, where I dived into more work more readily, just to see if the agent can figure things out on its own.
Busy does not always equal doing something useful.
Easy to pay a lot to the model vendor though eh?
Sure. So to clarify, no, I did not just spend it on busy procrastination, and I don't think it promotes that either. Would contradict my story anyhow, this was not some trick I was trying to play on you.
> Easy to pay a lot to the model vendor though eh?
Certainly, as I'm not the one paying. Though it's exactly corporate who really wants to have it both ways (who wouldn't?), and keeps trying to get me to use the crappy useless models because they are cheaper, despite them tanking productivity rather than helping it, so go figure.
There will definitely be a time when the hype dries up, and mgmt will start playing hardball. I'm confident that the value is there, and that I'll be able to demonstrate it to them that it is more than worth it. You can choose to not believe that, up to you. Maybe it really isn't true for your line of work, after all.
That said, the review burden is rough, that I can agree with. I outright felt compelled to evaluate whether the additional review burden did not outweigh the benefits, but at least for my tasks it did not. So grumpily, I simply live with that pain.
Maybe it helps if I mention that my line of work is DevOps and Operations. I have an ongoing suspicion that this area is better suited than average for agentic work. The codebases are relatively tiny, the languages and technologies used are very well represented in training data, and there's a decent amount of side chore. I can definitely imagine agents being a lot more frustrating to work with on proper, sizeable codebases, and the numbers simply no longer adding up. I don't have much of a first hand account with that.
My closest exposure is some personal toy projects, where getting the actual vision out there ended up requiring an inordinate number of turns (this is with a frontier model). In my estimation it was still worth it, but I definitely had to give it a back of the napkin calc.
As far as my work goes, it is of course not magic, creatively worded AWS docs will still trip it up (as they initially also do me). In those cases, my expertise is still required. But the well trodden is very well trodden, and I could cut out a lot of cruft, including a lot of organizational minutia, which I very much appreciate. I was able to burn through my backlog almost completely, for example.
That's why e.g. cross-generational learning rarely works. If culture or technology still allows it, you have to repeat the stupidest mistakes of your parent generation to learn the same lessons. "Learning from the mistakes of others" in your own generation equally hardly ever works. So much less if the lesson involves falsehoods you wanted to believe.
When I ask for their input, it's always for a situation where I'm capable of judging if their input is useful or not.
In all situations I use them, it doesn't matter if they're correct at all. I'm asking for ideas, alternatives, links for blogs or articles. I talk things out with them...
I don't think we should ever "trust" LLMs. This seems like the wrong usecase for them.
One smart engineer seems to have entirely offloaded all thinking and conversations to one, with just occasional editing. It's utterly bizarre to hold any conversation with him. It's one kind of rude thing if he was doing that to respond to me reaching out to him if he felt I'm not worth his time. It's a other when he's the one actively reaching out and asking my help with something.
When Linus posted that AIs and vibecoding were here to stay and declared resistance to it as harmful, I stopped to consider whether I was wrong, but it has made me realize that in retrospect Linus Torvalds and Linux itself aren't actually the holy grail of computing. I didn't feel that way with Richard Dawkins, its not like falling for an AI psychosis retroactively made me question The Selfish Gene, but now I'm looking at linux and the theory that it's a clusterfuck is gaining so much traction, especially after copy.fail and ensuing rustification, I see so much clearly now. It was never about linux, UNIX sure, POSIX, yeah, GNU fucking aye, kernel? Ok whatever, drivers and scheduler with a gajillion lines of code I guess.
1- Maintainers are allowed to commit LLM generated output.
2- Criticism of LLM generated code is not welcome/will be ignored.
Now, whether that constitutes being pro-Vibecoding or pro-agentic engineering, whether it's delusion, whether it will have problems, that's subjective. But I feel that whatever way you look at it, it's a topic that polarizes engineers, and Torvalds is taking one side and not the other. It doesn't seem to me that it's a very neutral stance, although it may be more neutral than projects like Bun or OpenCode of course, if it feels neutral, it's cause the overton window is shifting.
I would assume this is mainly that criticism that entirely amounts to 'this was written with an LLM' would be ignored. The actual quality of the code itself should be as open to criticism as any other piece of code in the kernel.
> It doesn't seem to me that it's a very neutral stance
Well, this is a matter of the window, isn't it? From my point of view Linus's opinion makes a great deal of sense, and is about as level-headed as anyone seems to get in this conversation. It's obvious that LLMs are useful. How useful, and for what tasks, and what downsides exist from using them, are all still in the mix, but the claim that LLMs are not at all useful for anything related to software feels like a very extreme claim to me at this point.
On to the technical point, LLM output is output, the source is the prompt, if you are going to commit something, commit the prompt. Second, code that is generated by LLMs is less maintainable, Linus entered late into the fad and anyone with 1 month of fiddling with AI knows that he will regret it soon, it's hard to undo once you corrupt your repo with slop, perhaps if it happens fast enough and there's no major releases it can be swept under the rug.
On to the nuanced point, Linux is purposefully designed to maximize user contributions, so accepting LLM contributions might well serve the particular purpose of linux, but I still think it's technically wrong to commit target code, only source code should be committed, and that's prompts.
But git itself is collapsing, it doesn't seem to be well suited for this new revolution, it doesn't track prompts, or it does so at the expense of the generated code. Maybe github can track target code as artifacts.
I think we are watching the collapse of Linux, Git and Linus. Certainly a bold position, so I don't blame you for being more conservative, but we can come back in a couple of months and see if we changed our minds.
Edit: Guys, why are we downvoting this? Does no one use like ChatGPT or Claude and understand how it works? Do you all think its regularly hallucinating links still? Is everyone on HN using like free signed out accounts or something? What year is it?
The popular answer is sometimes the wrong answer.
The other day though I was seeing how well it could pull details of its own conversations with me. It often does this pretty well for broad strokes of things - it remembers, largely, what cameras I have and use when I ask photography questions. It's never made things up here, but it does forget details, such as whether I've bought something or am just considering it. However, when I asked it for a specific interaction I thought I remembered, it gladly went along with my false memory and provided an affirmative answer. It was the first time I'd been caught in a serious hallucination with a frontier model (Sol High on the web chat interface) in a long time.
When did that stop? May 7th, 2026?
Thank you, may I have another?