There are a lot of mediocre programmers aren't there, I wouldn't be surprised if on average Claude is doing better than what exists. It's not a "lie", anyways, just a point of view.
I'm ok with vibecoding scripts, I love programming the machine and making the end user happy and not dealing with bash! I don't judge scripts quality as long as they are tools in the workflow, I dont care about that, unless they will somehow be part of the production. The name Script implies that, there isn't any architecture and Claude Opus sucks at architecture
For me, it writes code that mostly works, which is great. It insists on adding some of the least readable comments I've seen. It also likes to pad out the code to about double or triple what I would write.
It doesn't seem to have that little voice that says "hey, this approach sucks" so it needs the harness or the prompter to tell it to go back and refactor. Even Fable and Opus struggle with this.
I don't begrudge the business for making the prudent choice of making me use AI, I just sort of lament that what I spent years enjoying and getting good at was so utterly derivative that it was perfect for the LLMs. It would be insulting if it weren't just a fact.
Just like if you don’t read you will lose the ability to.
These skills are not innate to us and can be forgotten over time.
I wonder if it will get to the point where people don’t even know what to ask for. Or how to judge if what the machine produces could have been better. Or if it does what they think it does.
On the other hand, if you mean tight, performant, simple, well-factored code, then yeah. LLMs simply aren't there, and may never be. On the other hand, maybe that matters less than we thought it did. Time will tell.
To me it's just crazy of how dumb it is to test an idea with a vibecoded shitty app. I'm not saying it should be perfect from day zero, let's pretend we are talking about cakes, how crazy it would sound you will tell people to try and judge your new idea of a cake you don't even know how it taste or what's inside?
For the first group, it doesn’t matter how good your cake is. If you spend a year perfecting it, you won’t sell them any more cakes. And for many products, this group is basically 100% of customers.
It's disappointing that the prevailing sentiment now is that architecture is less important than we thought. Software is always going to be getting more important, and all of it should be well thought out. If these people had these tools in 2004 they might've made thousands of fragmented buggy social media startups and actually slowed progress on that front.
I completely agree. The fact that the software from places that are very vocally pro-AI (microsoft, salesforce, etc...) is still so full of bugs is really contrary evidence.
I would recommend "The Leprechauns of Software Engineering" by Laurent Bossavit for an insightful and entertaining overview of just how much of software "engineering" is based on "truths" that we don't question.
Those weren't lies. The AI does produce better code than those people. They simply weren't very good programmers to begin with.
I think this goes a long way to explain why there is such a huge difference in opinion about the quality of AI-written code: Some people say LLMs write code far better than they could, and others say the code is always terrible and requires a great deal of fixing before it's acceptable. How can two different developers say those very different things about the same product?
It might be a tough pill to swallow, but if you think your LLM is producing better code than you could produce... well, I might have some bad news for you about your own skills.
But in the new environment of LLM-assisted coding, the LLM produces acceptable code so much faster than I can write it. Just taking time to understand and retool the LLM's output means an order of magnitude lower productivity and it doesn't feel like that's acceptable in this era.
Folks I know are landing 10+ PRs a day and nobody is reviewing anymore, and these are not companies that would have been skipping code review 2 years ago. The whole culture is turning against taking the time to comprehend and reason about our own work.
There is a wide range of code that will do the same job. 15s vs 2h for code 60% as good? 10 minutes for code 80% as good? 10 minutes today and 10 minutes tomorrow to fix something that is 30% as good?
A lot of people are going to draw that line in a different place and now that people can make that trade off you will quickly find they will and it probably wont fall where you would like it to.
At the end of the day a business pays you to produce value that will earn them money / customers. They don't pay you to write good code. Sometimes those two align and both sides are happy but they don't and often wont.
I wonder, given LLM's ability to "produce code faster" are they also producing value faster? I don't see it as an end user. Software is getting worse and worse, buggier and buggier, every release. I don't see it at my company either. We're producing massive quantities of code constantly, but still every project is on fire and every line of business is way behind schedule.
I don't know. Maybe.
Software might be getting worse but we are not far enough along yet for it to matter. Most people use less than 10 websites total, the internet has shrunk. It will take a while for it to be obvious in the other backend services that they dont know they use. People are really tolerant of buggy shit and some software has already been basically wiped out. Dev tooling is a blood bath right now, things like tailwind and all sorts of little CI tooling is fast disappearing. I wonder what happens to the open core stuff like elasticsearch / gitlab / redis where they gate some features in the enterprise offering that could probably be implemented by LLM pretty fast.
> We're producing massive quantities of code constantly, but still every project is on fire and every line of business is way behind schedule.
Right but did they start altering the schedule when AI entered the picture? We have barely just passed "tokens == productivity".
Also people tolerate quite a bit of buggy shit. Anthropic is down to two 9s (aha) and people still use them. There is a github ci failure post on here like every other day.
"The AI labs are scared of their own models, surely they must be credible sources on how powerful those models are"
"They are asking the government for regulation, they must be even more responsible than the average company"
"They are claiming to be a software engineer who automated their job with AI, surely I have to buy their course"
There are many boilerplate pieces of software that an LLM can generate faster than a human, but nearly half a decade into the LLM era there is no evidence that it's leading to the apocalyptic end of software organizations. And that's all with the subsidy in place that is now disappearing. Everyone online is lying for money, and that money is coming out of the same investor-subsidized marketing budgets.