This is also the best way to understand a codebase, and it's quite enjoyable.
I think the models are getting too egotistical. They're so confident of their fixes they won't bother suggesting basic techniques like isolation of the problem by disabling code paths, for example. They'll keep shotgunning less-and-less likely fixes with undiminishing confidence until the code is full of fixes that do nothing. Then they'll double down on why these should be kept.
EDIT: don’t get me wrong. I still think AI is incredibly useful for a lot of tasks! But when implementing an architecturally hairy thing, I find it less stressful and equally quick to jump down to the editor level and use AI just for code completion.
If you think about a product like marble: it's something that most be chiseled out of time.
Some people can chisel better products: the AI is just a better chisel.
Sometime still has to guide the chisel and judge the art/product.
In our cases, the market judges products.
Beyond that I don't trust it at all and I still write all my code the meat sack way.
Trust is earned not given and it hasn't earned it yet.
I for one welcome the fact that this whole thing has driven me back to books and deeper into the fundamentals. I have never read so much on math, hardware, and history as in the past 3 years or so.
And if it’s just for enjoyment, then doesn’t it make sense other people who want the same would just get a job where they can keep pushing things to an AI, feel good, get paid, then quickly get back to the hobbies they really love?
Maybe unforeseen breakthroughs and different architectures are achieved. Given LLM fundamental shortcomings grounded in mathematics and information theory, I highly doubt they will and we will always need to deal with these issues in some capacity.
Do your silver platters give you some competitive edge? If not, then is the ego problem yours, or your coworkers?
For me, as long as I'm properly RPI looping it and not blindly pressing "yes" then it will nearly always reach the solution, usually a fair bit quicker, because it effectively becomes an ideation machine that can keep more thoughts and knowledge in it than my brain can.
However, I'm using it through giving it the context, it has access to the repos, access to the sytems, I'm telling it where the logs are, I have docs to show it what each part of the system is for.
Along with that, it does depend on my own instincts/knowledge for me to read its response and for me to say "no, you've over-thought/engineered this and this is actually the better solution", but its very are nowadays for me that it can't find the 1 liner, as long as I've fed in the right context, mostly pre-done because I've spent a bit of time building out the context tree for the repos/systems in a way that allows it to know what it needs to know.
I'm not doubting that people spin their wheels and couldn't find the one liner, but if its as simple as you say, that does seem like people who aren't great at LLMs along with a lack of instinct/experience.
I guess in essence, I think you can use LLMs in an old school troubleshooting way, and I find it still speeds that up the majority of the time. Its basically how I use it most of the time. And like old-school troubleshooting, if you build out the LLMs context over time, it also grows in capability, as long as its being used as a tool and not blindly trusted.
Should I not assume that most senior developers who are serious about LLMs do this?
Usually, I take my time to understand each keyword of the code I'm looking at, especially if it is new to me, like terraform.
I work in a team/with one architect, who only did the DevOps/Infra stuff for the past years and I had the expectation he knows what he is doing and talking about.
At around 2 weeks, I noticed how his knowledge has severe gaps and how he takes things at face value or uses terminology interchangably, which confuses me. It sounds plausible, but it does not actually translate into a working system or shared understanding.
Then one day I did some pair programming with him and whenever there was an error or a resource missing, he would type it into the LLM, copy paste it out of it and then brute force error messages. He did not even wait a second to think or reconcile whats happening on the screen or what the exact requirement is. Never taking one step back and questioning any assumption.
Now that the timeline is shifting and everyone starts to be stressed, he continues to vibe code through me and it is so tiring, there is no higher level planning or architecture, its just a reactive type of trial and error to be faster. It feels like these people are so used to talking to bots, that they treat you like an agent they can chat to or talk through monologs with.
It is quite shocking how people went from being humble (learn the basics or close the gaps in understanding) to full on authority on everything and berating people 24/7...
So right now I'm considering quitting IT for a couple of years until people calm down, but I think its pretty futile
At my place, this is what they want. They want people to smash through things as fast as possible. They don’t want people to sit and craft a solution which takes in to account the whole. They are choosing tools which are low code, and use llm’s to produce what they need. as they say “this is the way things are going”.
The kind of person who insists on understanding things and working through the problem has always been rarer. It's not "humble", it's "inquisitive" and "persistent".
the less friction the less growth.
So even if they copied from SO they would have better knowledge than an Ai user.
This isn’t speculation either, theres an MIT study which this is based on. https://www.media.mit.edu/publications/your-brain-on-chatgpt...
That was never humility. That was fear. This technology is just enabling people to show their through colors. The unearned confidence is a hell of a drug.
So quit pair programming. I never did, never will do that, nor worked at a place that remotely encouraged that. Each to their own, that's how it should be.
The speed AI debugs at is incredible and yes, we lose touch the more we use it like any manager feet up barking orders to their underlings to get things done.
I think what's happening is ... poorly aligned and led engineers [in] this ego inflation feedback loop where they are completely detached from reality
A story about a team of humans with some very human problems.
I guarantee you'll spend less on tokens, have better documentation, better code, and most importantly more competent engineers.
That's the other side of the country, but it's how it's done over here also.
Still pull the old plough varieties, still practice shed blacksmithing, etc - even while developing and using autonomous Ag-Bot tractors for spraying, etc.
On the surface it seems rational, but it quickly turns into a system of perverse incentives because now the development team must maintain an illusion that they are constantly overwhelmed with tasks and could never hope to spare a microsecond to assist the customer. This misalignment is how you wind up building your own web frameworks and databases from scratch. It turns into a self serving monster that eventually dominates the entire business. From the perspective of the business, many of these development teams look like they're behind some modern day iron curtain.
After three decades of outsourcing to lowest bidder, I do not buy that humans are somehow better!
> patient, old school troubleshooting
I usually see similar arguments around systems with major red flags (no docs, poor CI, decade ago no CVS...). And engineers with private stash of workarounds for job security!
Claude does not do anything special.
Or perhaps claude was misconfigured, it had no access to relevant part of system, and it tryied to work within its limitation. Often it means decompiling binaries in desperate loop...
Overabstracting, deduplicating things that don't need to be. Building metaclasses because it saw a single orchestrator in the whole codebase.
If it is a better engineer than you... You need practice.
In a way all we've done (currently) is drastically expand the amount of technical debt across the whole industry.
Should be profitable for the ones who can still actually program though and haven't let their skills atrophy by letting Claude do everything.
I don't deny there are use cases for LLM's I'm just don't buy the hype about them either.
As with all tools, you have to understand how to use them to get done what you need to get done without sticking the chisel through your hand.
Anecdote I like to tell.. I was working on a financial planning software, intentionally purely vibe coded as an experiment.
I eventually discovered AI had implemented seven duplicate copies of tax calculation functions. All of them different. All of them wrong. All of them giving different answers for same input.
Not even the most junior of newbie junior engineers would do something this crazy. But AI was happy to do it. It will solve the immediate problem, efficiently. Even if the most efficient solution is something ridiculous like this.
A person that was hired as an expert in our startup spent more than one week full time working on implementing his solution to the problem we were having. I checked the code after one week to see the progress and was curious how they are implementing an already crazy sounding idea. I found that the whole week was spent re-implementing in python, python's built-in "float" function. That was it, the whole code was just that.
Our problem was related to financial services and their implementation of "float" was not even correct.
The llm just allows to generate faster.
We can feel smug about that but all it means is that we need to be clearer on our requirements and preferences up front.
State that similar functions should be in one place and there should be only one. Today there has to be compelling reason why that function is different from others. Normalise the function name based on what it does. Why are there different ones?
Then there are all the other guard rails in place.
Better guidance from mentors, reviewers, and automated project tooling helps everyone. Juniors, seniors, and engineers.
And whilst it is obvious things are growing... Saying that, sounds almost entirely like the person saying it cannot objectively look at the environment. If everything has changed in the last six months, why has the industry not radically changed to match it?
Everything really did change with the Pentium II. It did with 3dfx. It did with Damerau's taken on Levenshtein. Hell, everything changed with React. The AI leap with seq2seq completely revolutionised the entire industry. But... Its kid, the LLM? Really?
How many times I heard ORM are bad only to recreate the same shit?
How many times I heard ORM had bad performance and see 1+n stuff everywhere?
How many times I have seen tight coupling in the name of DRY?
We don't get seniors, without juniors. I'd say more than half the job, is just... Learning. People grow.
But that is not a point. Claude can code tight compact loops, it just needs to be instructed to do so! If it does "enterprise code", it means it had no instructions about code style.
If your documentation, spec, agent.md does not have proper guidance on coding style... yet another red flag!
Considering how often it overrules, its own rules?
It is my pet peeve with Claude and why I don't prefer it for most stuff
(also the comment spam - but that's a all of them in a way or another)
Some asian devs will go for weeks without asking for help, all while giving amazing fake status reports. Loosing face etc...
https://x.com/PawelHuryn/status/2095982259761475945
https://bughunt.productcompass.pm/?preset=all
Claude Opus 4.8 ranks near last on this Bug Hunt benchmark, and missed 96% of the deliberately introduced bugs. If you're a developer who has been falling back to Opus 4.8 because of how Opus 5 talks, and Fable 5 being so expensive that it needs to be rationed... well, turns out Opus 4.8 can actually be quite poor for finding bugs.
(Which feels weird to me, because Opus 4.6 fixed a bug that myself and a group of humans had been hunting down for over a decade. Models are spiky.)
Also surprising to me: Luna Max performing better than Fable 5.1 High, at least on this benchmark. But Astra 6 & Fable 5.1 on Max both perform at the top as you would expect.
If model can not automatically reproduce bug, while human manually can... you got a problem in CI.
> Luna Max performing better than Fable 5.1 High
Perhaps you are reading too many benchmarks.
Edit for answer :
I agree Luna is great cheap model.
But if Fable was hitting security limits, yet was still included in benchmarks...
What flies better? Elephant or paper plane. You can make objective benchmark about that. But not much value for logistics company
Maybe, but at least the benchmark provides an objective measurement of the codebase it is tested on. You're also assuming the bugs are newly introduced / regressions.
I can give a concrete example - Fable will not interact with bugs that result in writing to null pointers in C code. That triggers the guardrails and ends the session. If Luna (or GLM Flash, etc) will fix those kinds of memory bugs, that immediately puts it ahead of Fable in some ways, no matter how tiny Luna is. Again, models are spiky.
I still agree with your initial point! It's LLMs all the way down over here. It would need to be a particularly gnarly bug & an exceptionally talented human for me to want to pay another human to work on fixing it now.
Hard disagree. We absolutely cannot assume that. You can posit it, and we can have an informed debate about it. This is what irks me the most about LLM fans: they constantly try to reframe the debate to have their worldview as the agreed-upon starting point.