What have we gained aside from shifting where time and effort is spent in the design loop?
The verification is what we always wanted but never had time for. For 30 years I wrote side project games without extensive testing or a fuzzer or anything because I wanted to spend my time writing the game. Now I can do that and still have the testing I always knew I should be building.
This method also helps with the classic problem of being unable to identify essential complexity up front.
Right now, we're in a "honeymoon" stage, where non-engineers think that they can somehow, magically produce stuff that is comparable to what experienced and disciplined engineers can do.
That won't last long.
Despite all the caterwaulin', LLMs are just another engineering tool, and good engineers will figure out how to implement them, in ways that absolutely blow away the garbage produced by non-engineers.
It's like that rich, no-talent guy that buys a PRS guitar, and thinks he'll beat the experienced musician that still uses his beat-up old strat, but consistently makes hits.
- code? up - build budget? Up - developer output? Up - developer oprrational costs? Waaay up!
Why is this Bad? Every one involved profits somehow.
I feel like your suggestion would be a nice organic extension of that same practice (at the cost of more infrastructure/tokens)
If someone finds it burdensome to review my 50-line AI-generated bug fix, adding 200 lines of tests only gives them more lines to review.
I have no shame in accepting vast quantities of ai generated automated tests at review time without really paying attention to them. It’s better to spend more time on the core implementation.
I respectfully disagree. IME writing a good test suite is one of the areas where current LLMs perform worst. They certainly like to write plenty of tests but those tests aren't necessarily adding much value and they don't necessarily cover the important functionality.
Given how heavily LLMs and specifically agentic workflows still rely on producing something vaguely correct and then iteratively applying guard rails to reach something that actually works I humbly suggest that trusting a very substandard test suite to be one of those guard rails might be unwise.
I realise this is heresy, but my conclusion is that unit tests aren't important any more. E2E testing on the other hand...
(I'm agreeing with you about the quality of LLM tests, I'm just trying to make lemonade out of all these LLMons).
All those extra tests are churning through tokens and CI resources. These are far from free in a direct financial sense. I've also noticed some of the more recent models will try to do a lot of experimentation and ad-hoc testing by spinning up shells or running something like Python scripts direct from a command line. Every time they do that you either have to review whatever they propose to run carefully - which is slow and tedious work - or stick the tools in YOLO mode and hope that your agents don't accidentally wipe your entire filesystem, open your firewall for malware to take over your network, and perhaps launch a few nuclear missiles.
I would respectfully disagree about the value of unit tests as well. The reason unit tests appear to have less value in the LLM/agent era is because current LLMs suck at decomposition and other architecture-level work. They just see a mass of code and change it as a mass of code. This probably results in orders of magnitude more code being written/changed than would have been necessary for a skilled developer working on a well-maintained code base to solve the original problem and that automated work also takes longer in wall time. Again this gets brushed under the carpet because a lot of people trust that LLMs are so good that their costs are negligible regardless of how much work they do. People used to say that about the Cloud transition as well because not needing so many in-house staff to manage your resources was going to be a huge saving. Today we have companies bringing in consultants because they literally can't even work out how much they're spending on AWS by themselves. Instead of having a few greybeards who could make a Linux server do anything we have a few increasinglygreybeards who know which combination of 17 AWS resources they need Terraform to spin up so they can write the web server equivalent of hello-world.c. Meet the new boss - same as the old boss. :)
The goal is to get to a point where I can say with a good level of confidence that a change didn't break anything. It might be the wrong change, and it might be crap code, but I can ship it without anything blowing up. I've been using it for a while across a few game projects and so far it's worked quite well.
Testing is NOT verification.
EDIT: Sorry, oops, it's also called a kind of verification.
But there is the problem that tests only check or verify a small sample of possible cases or inputs. They can never show that the program is correct. (Famous Dijkstra statement.) To know something is really correct you have to understand and prove it.
Black-box e2e testing isn't suitable, or practical, for a lot of applications. Unit and integration tests generally require you to understand the code you want tests for, to know what to actually assert
Are you just trusting the AI to do it all?
Even if your test suite tested every single possible behavior and code path, tests usually can't intercept all possible unintended behavior that an LLM might have added.
In all likelihood, most test suites don't intercept any unintented behavior (such as unintended internet connections).
That's not a big problem when a honest human has written or reviewed the code, because unintended code is usually easy to spot.
But an LLM might have dropped any whatsoever code: maybe some of your project's code resembles the one of a trojan in its training, so it completes it with the trojan.
Or maybe it found that if your server is unreachable, it can upload the client's data to a public Github repo; or that if there's no space left on the device, it can delete some of the user's folders, easy!
Would your clients be understanding if that happened?
Eventually, the tools become trustworthy, and we no longer need to worry about their products.
We're just about there, with frontier LLMs, but I don't think we're all the way there, yet.
English isn't a good programming language if you want to trust that the output will be as you want it to be. High level code has no room for ambiguity - that's why we use it. It's a concise, very explicit language to tell the compiler exactly what to do.
Dealing with an LLM is a fundamentally different thing. The reason it's able to speed us up is because it makes assumptions for us and reads between the lines. The only world where we trust the output is one where it can literally read our minds. That's not coming any time soon