Most often we’re satisfied with signals and indicators that it works and we build theories and mental models that use to try to avoid the effort of proving anything.
Worse is what happens when you start using LLM models to generate the code for you. I find you start losing the mental model short-cuts; you start to trick yourself into understanding the code. You understand little.
And what few studies there are of code review suggest that we have very little impact on code quality after reading more than a couple hundred lines of code over an hour.
So the current mental leap is, “the AI is a better programmer than I am.”
I think it’s still worth becoming a better programmer regardless. And there’s no royal road. You just have to write code and struggle with writing bad code.
This is not just a software problem. I see it everywhere. Folks in business roles just as much as any other role. There is a surface-level operating framework to get a job done but the why/how is sometimes lost.
I always come back to an example in my past. Somehow automated a report. From Excel to Power BI. Then for months Power BI was wrong. Automation (more code involved for sure) made it run faster but then people forgot about the how - how to check/validate/etc. And since it was code - for many non technical people the whole thing was a lot harder to deal with needing to deep investigate issues.
For everything else the general mode is "can we make it work well enough". And when it breaks, well, we will curse and pry it open and either fix or rebuild it. There are many stories of a particular thing that was built long time ago by folks who are long gone and no one knows how it works or even where it physically is.
And it works fine; usually better than ensuring complete reliability from the get-go, when it is not clear if it is a long-term core feature or a kludge that will be obsolete soon. My 2c.
But I feel like it's a minority that really know how the software they're writing really works. And it's 10x worse for older legacy software.
Mostly because the people doing it don't care. Not judging, just reporting I guess. I know because I used to work in quite a few companies where the "money maker" app was on maintenance for years and nobody was able to answer questions or didn't even know most corners of it still existed.
This desperation mindset also often correlates with people writing code without care for the next maintainer.
And the scary part is that this also correlates with advocacy for going all-AI.
I don’t know anything about formal verification myself! But proof? I am sure we have seen that AI capabilities exceed humans on that front.
I recently started using Claude to write an app. I was going to write it by hand but was having trouble getting motivated, and I knew I had to "up my game" in the vibecodery anyway. So far it's been a lot of back and forth, I keep the leash pretty short because I don't want The Claude AI you see everywhere now.
But even with good intentions and the short leash, I find myself saying "yeah, whatever, it looks like it did what I told it to do" and not actually reading all the code. Some yes, all no. And this is for my own project, nobody is my boss here, and it's nothing I couldn't have done by myself.
Now imagine the incentives for someone who's doing it for money, whose boss is pushing them to do more AI and not less, who probably doesn't actually know how to do all the things the AI is doing for them, who looks around and sees who's getting promoted and who's getting laid off...
And remember that most of us, most of the time, no matter how much we might love computers and software, are that person.
But hey, it could be worse! Have you heard about the Literacy Crisis?
I used to think this but have come around to the opposite. I'm already seeing client projects where they went off and LLMd their way forward. A few hundred KLOC and many rounds of bug fix whack-a-mole later, they have something that's a giant house of cards.
Sure, once in this state the LLMs can repair issues at a faster rate than the humans can, but the damage is already done at that point. And it requires a lot of combined human/agentic effort to unfuck it.
That stops working once the code reach a certain size/complexity limit. It is a different limit for different people, but there is a point where there is just too many parts to be possible to fit in a head. There is also a middle stage where everything fits, but can't be accessed all at once but require some time to retrieve specifics when looking at areas not in recent memory. The middle step is more achievable the easier the code is to reason about by black boxing parts.
Yes, but it's not ever too much code for a team to understand, or a community. The question is whether you can build a team or community with enough trust and integrity that two things are always true:
1. Alice understands the fluxtron renoberator module she wrote.
2. Bob can also shift his attention there and understand it if necessary, because Alice writes clean code that was reviewed by John and Sally.
You can have Alice, Bob, John and Sally all be robots, and you can decide to trust the robots, but (so far anyway) it's a choice, not an inevitability.
Every large project I ever worked on worked exactly this way. What do you mean it was rare?
I'm ready to bet that is untrue. Sure, no one can recite the code, but people can quickly scan the module and refresh their memory as well as understanding the new changes, even after a while. If not, collaboration would have been impossible.
> Because 3 months later, the fluxtron renoberator module has been modified, fixed and refactored by 5 other people and looks little like Alice remembers it
I don't know for which type of projects those things happen, but good design can have a long staying power. Drastic refactoring in a short period is a clear sign of bad engineering.
> and Bob, well, he could never shift attention and actually understand it, nor could anyone else, because that's just a nice story we told ourselves that never worked in practice.
A lot of OpenSource projects are decades old, and while some core maintainers can stay for the lifetime of the project, new contributors came, understood the code, made changes, then left. Same for hires in companies.
I don't understand this part. When I was doing freelance consulting, my job was quite often "come to our office, scan this 40 million lines codebase and fix stuff for the next week". It was never a problem to read and get up to speed completely unknown codebases if you have basic programming ability. Much harder to do something that fits floaty business objectives though.
Perfectly and succinctly said.
I can’t help drawing a parallel with what is going on now. Even though I’m pro AI, I understand why QGIS is stepping back. There are a lot of places where the processes and tools are not ready yet to fully embrace it and you need time to prepare.
The way I like explaining this to people is that I don't think I build products anymore. Now I build the harnesses which build the product. And a massive part of that is ensuring that I can make the AI remain aligned with my way of thinking.
The software telephone game seems to never end!
That’s the old “Programming as Theory Building” stated by Peter Naur. Writing code, once you learned a language and a few libraries has always been easy and can be made easier by using tooling. Having a good understanding of what has been written and what needs to be added is the difficult part.
I think for every line of code I’ve written, I spent 2 to 5 hours reading specs and docs. Creating software is always about understanding. If you’re thinking too much about syntax and libraries symbol, you’re just at the beginner phase of coding. Like thinking about grammar when learning a second language or being hyperaware of the car as a beginner driver.
The bottleneck has never been the writing of the code. It's all the rest that takes the majority of the time. Of course, it is often the writing which lets you expand your understanding or your awareness of challenges and pitfalls, which then puts you back on the drawing board.
Ah, that's only true if whoever is in charge is willing to _have_ a bottleneck in reviewing. In some software development groups, there is a strong pressure for reviews to pass, i.e. expectation that reviewers approve; and the notion of veto is not common, so that if some reviewer objects, the implementing developer can find others who would be willing to accept, bypassing the objections. Also, few people are willing to conduct long reviews of a lot of code, and even less people are willing to read multiple comments in a long review of a lot of code.
Now, in some projects, there are only one or two "owners" or "marshals" for some areas of the code, so they can apply a strict standard if they want to, but - their time is limited, meaning that long LLM vibe-pull-requests are likely to either be ignored for a long time or rejected off-hand.
I have many years of the experience of of raising concerns, criticism and objections in reviews
I’ve mentored juniors and the common trait is that they rush to code without understanding, then they spend days trying to fix things. Coding will seem like a real bottleneck for them.
As a senior (of sorts), even though I don't quite rush to code, I often find myself starting to code something 'naively', while my understanding is lacking (typically in a tangled, inconsistent and poorly documented codebase). But what probably sets me apart from juniors is how I handle getting stuck, or rather what it means for me to try to "fix things". I spend time thinking about what I've discovered that has gotten me stuck, what I must take into account, and how I should alter my approach (or refactor the existing code to be more accommodating).
Sometimes I manage to formulate ideas apriori that reduce the frequency of my getting stuck, and/or the extent to which I need to backtrack. But - not always.
Have you ever heard of the V model (https://en.wikipedia.org/wiki/V-model) ?
Long story short: it does not work, because the act of coding adds new valuable inputs that alters the (original) design.
One might say "you just have to know", which is an equivalent to "you just have to figure out the future", "you just have to be perfect" and other non-sense (which does explain why V cycle does not work, high inefficient, low quality outcome, bad method)
People that do trial and errors lacks direction and their code is often fragile, especially with evolving situation. Because their have failed to grasp the deeper truth of the domain. The usual signs are tightly coupled code and code churn.
Honestly, where did the bottleneck end up now ? It sure isn't simply reviewing, or designing, so where is it ?
Assuming you're comparing to experienced human developers that are good at what they do, then I think you have to define what "better" means here. It's clear that you can get correctly functioning software faster in many cases, but it's not clear that the design will be better in terms of conciseness and readability or that it's as useful as a foundation for building upon later if nobody understands what's going on inside.
But to me, having graduated last year, it doesn't seem far-fetched to think that AIs will also get better in terms of design.
And will conciseness and readability even matter in the long term ?
It's all things I think about more and more often and I'm not sure where I will end up
Fair question. Right now the way for humans to understand code is to read it and that'll be easier if the code is concise and readable. But it's possible that we'll end up in a situation where the code is separate from the explanatory materials that humans need. This is similar to math where we might have impenetrable Lean proofs of theorems and then natural language expositions to explain the proof.
If I can shit out some vibe-coded thing over a weekend that does <useful thing> it means anybody else with the same tool can do the same thing. There is no value in the actual code itself -- the faster these models run, the more true that statement will become. In the not too distant future these things will run directly on silicone and operate at thousands of tokens a second. That means they'll iterate and explore something far faster than they do now -- mistakes they make will be fixed in seconds or perhaps milliseconds.
That being said my <useful thing> is likely only "60%" of what makes an actual product somebody would buy. It's finding the other 40% that is something I don't think that LLM's can solve. That requires human judgement and then knowledge of how to communicate that vision to your agents.
In short, I believe the challenge is knowing what to build that others would find useful. It's easy to crank out <useful thing> for yourself. Making something others can use is not. And that is, possibly, the bottleneck. But once these are fast and dirt cheap, (and that is a when not an if) the cost to iterate to something workable goes way down. Of course mumble mumble "no silver bullet" / "mythical man" month mumble mumble and all that soo......
I dunno... doesn't directly answer your question but it is what came to my mind anyway.
My prediction is that in the future we are going to be more focus in the testing part (that should be more human readable) than the code itself.
Whatever it is, the only thing clear to me is that the tooling will have to change.
I can't help but feel that when I mimic their behavior with an LLM, it works great. Just keep pounding claude saying what I want, prove it with tests, and have claude give me a 1 shot script to build and launch the app. I get hands-on time with the UX a lot more (I think?) and claude happily iterates. Most people stop happily iterating after like iteration 2; then becomes teeth-pulling.
So to your point, claude's tests are gibberish but the 'testing' via user interaction is what I think the future is going to look like.
We will need some novel systems where hot-reloading is the default experience and you are looking at the build output constantly as opposed to the source.
claude was like "I took your logic, removed the shared state, ran the tests 600 times in parallel, proved no state is leaking across runs". I was impressed...
I am doing an app that keeps a web and desktop version at the same time, mostly because the agent can iterate faster and with less resources and more constraints (running in a container) with the web version.
One approach that Ruby took for type introduction was to put that in a different file. That comes to my mind.
Typescript brings some niceties, but also some boilerplate all other the place. Combined with something like Effect, Zod, and the like and line count explode. Not sure OCAML or Haskell would provide more compactness for equivalent safety.
There was no mention here on rushing to anything. Verbosity is not always bringing actual non-duplicating information. And even when the explicit way is taken, it can be put aside from human main analysis observation point. Actually even Typscript allow to organize type information by using .d.ts files for declaration files that contain only type information, while implementation files use the .ts extension.
Then there is the tradeoff between grepable explicit type and spare implementation with as much type inference as possible.
Is the fear that it is (maybe wilfully) misinterpreting the code, being too general, not going into specifics or is just not smart enough to catch all the nuances, privacy issues and hidden mistakes? Is it a token issue? A combination of all this?
> An eigenvector or characteristic vector is a (nonzero) vector that has its direction unchanged (or reversed) by a given linear transformation.
Do you understand eigenvectors now? Speaking for myself: certainly not. To really understand even that definition I'd need to do the calculations, play around with how they can be used in different circumstances, and allow my brain time to sit with all this new information and become comfortable with it.
I think LLM code descriptions are similar. They don't really bring understanding. Sometimes that is fine.