This wave is largely bugs found in pre LLM code. I'm sure there will be a smaller wave of bugs from llm code from the sonnet 3.5 - opus 4.7 era, and peeople will always write sloppy code/use dumber models under time or cost pressure, but since Opus 4.8/GPT-5.5 xhigh LLM code stopped being intrinsically worse than human code. At this point if you're shipping llm code that's lower quality than human code that's either because you don't have the money to spend on frontier models or because you actively choose a lower quality standard because of either various pressures or just not caring. if it's the latter your human code under those same conditions would also be really bad.
replyAstra writes a lot of security bugs too, the upside is that if you also use it to review the output it will find a lot (most?) of them.
replyThis is one of the least talked about and most useful features in any harness I would say, the review feature. I have never not found issues (either security or implementation) running it, no matter the code base or language.
My job is seeing a similar deluge of security work and >90% of the bugs have been in pre-2024 code, and zero of them have been bugs introduced after we were first officially allowed to use LLM codegen.
replyThat could also mean the LLM is blind for the bugs it procudes itself. Just like any ordinary programmer would, on the job.
replyTheoretically yes, but empirically that's not the case. All models I've tried are really good at finding bugs they wrote themselves, especially in a fresh context.
replyConsidering the software is much older than LLMs. Not many.
replyAlso, in the article they mention a significant number of patches for older versions as well.
Software is incredibly complicated. When you get into areas like security and networking and the internet its even more complicated. We rely on layers upon layers of software and hoping they are all done perfectly with no issues. The amount of junior devs, or outsourced devs for $10 an hour working on security relevant code is very very high, and always has been. LLMs if used correctly should be raising the floor.
replyThis wave is largely bugs found in pre LLM code. I'm sure there will be a smaller wave of bugs from llm code from the sonnet 3.5 - opus 4.7 era, and peeople will always write sloppy code/use dumber models under time or cost pressure, but since Opus 4.8/GPT-5.5 xhigh LLM code stopped being intrinsically worse than human code. At this point shipping llm code that's lower
replyAn uncomfortable truth about the era of LLM code is that machine produced code is, on average, quite a bit more secure than human code.
replyUnsurprising to those of us that has been screaming from the rooftops that human code reviews were barely catching anything in your typical corporate environment. The majority of the value was pointing out missing tests, because breaking bugs got past human eyeballs on a regular basis, even in very good companies.
replyIt's not that the LLM code is that great, but that many overestimated the human code and the human review.
I think it's more likely to be the same as average human-written code; which is what they were trained on, after all.
replyNo it's far beyond average human code, and it's trained in RL environments on synthetic data etc now. The average human can't solve a millenium problem in mathematics but the LLMs are able to now. Anything with a verifiable output can be RLed on to make llms superhuman it seems, and the average human is not as persistent and thorough as e.g. gpt 6.1 sol at checking every edge case and hunting for bugs because that behavior is something that's just trained into the model. Frontier models perform farm above the average human at the categories of tasks they are RLed on like code and math.
reply> The average human can't solve a millenium problem in mathematics but the LLMs are able to now.
replyI think comparing the resources they spent at it would be a more apt comparison.
AKA if you tasked 150 mathematics departments to only focus about that problem, and stop doing things like teaching and researching anything else, you might get close to the amount of resources that these companies put into solving the problem to get a press release.
yet LLMs dont do off-by-one errors, typos mixing up variable names, interesting they seem unable to learn these human patterns
replyThey definitely write off by one errors. One of the first serious work tasks I used AI for a few months ago immediately had an off by one error.
replyWhat they are really good at is writing boilerplate and reading API documentation / example code. Avoiding subtle errors is not what they're good at in my experience.
I've seen both of those (and others besides). I don't know how much code you've looked at but your negative directly contradicts my experience. Review that code. All of it. Be super critical because it looks good but that doesn't mean it is good. Those are two entirely different things.
replyAlso: most tests are positive tests. Tests rarely observe the negative space: that which is there but shouldn't be there.
They do those things all the time. More often than humans, in my experience.
replyCitation needed.
replylooks at particledb
Do you mean "created" as in LLMs were used to discover existing vulnerabilities, or as in LLMs wrote unsecure code? If it's the latter, then the answer probably is "very few". Frontier models have been better than the average programmer at writing secure code that follows up-to-date best-practices for some time now.
reply