undefined

points

[-]

I dunno about holes, but…

SQLite has gone into a frenzy of bug fixing this past year, changing long stable processes to ease things up: opening a specific report channel just for these kinds of bugs, adding features to Fossil to help manage things on their end (and that's just the visible stuff).

And with all that, the latest release has unfortunately been the buggiest I've experienced so far.

I skipped the .0 because of a crash, and only released .1 after giving up waiting for a .2 that never came, with a patch to fix a serious performance regression.

I spent a month supposedly shipping old bugs no one had a problem with, rather than shipping new bugs that instantly crashed things or caused measurable regressions.

That's on a software with stellar testing, and a good engineering team known to favour stability over the new shiny.

by Cthulhu_1 hours ago|

prev|

[-]

I don't think this is exactly true; adding holes only happens when adding features, and while LLMs can add features fast, deciding what features are actually wanted and needed is still a human factor.

So the amount of new code added is limited by processes and product management. Add to that that newly added code can get a lot more checks on it now, and new code will be more secure too.

I also don't think your main premise (adding holes faster than we can find them) holds up, but that's another matter. TL;DR I think we're in a high intensity period right now, but this time next year it'll be quiet in comparison - possibly quieter than before AI code scanners were a thing.

by cloudbonsai5 hours ago|

prev|

[-]

Do you think AI is as good at writing bug-free code as finding bugs? If that's the case, this should be a one-time event during human coders -> AI transition. It's like catching a lot of bugs during a handover review.

by wongarsu2 hours ago|

parent|

[-]

The code it writes in the first attempt is at least as buggy as a human's. So the question is more whether it is as good at spotting bugs in its own code as it is at spotting bugs in human code

I'm tempted to say "no, it isn't". But I don't have numbers to back that up

by jiggawatts4 hours ago|

parent|

prev|

[-]

Most software used to be written in assembly language. Microsoft DOS was mostly .ASM files, for example.

That era is over, compilers are used for almost all software, eliminating entire categories of bugs.

by jimbokun8 hours ago|

prev|

[-]

It feels like we need AI methods to write provably correct code, along with a way for AI to automatically translate existing code into corresponding provably correct code.

(And hope performance stays in the same ballpark as the old code.)

by bredren6 hours ago|

parent|

[-]

Yes. Allowing a naive agent to fix bugs can quickly become pollutive.

I suspect both old codebases and new ones that have a very intentional architecture are both dangerous to let loose.

The agent doing the fixing must have not only an incredible map of the project and how it is put together, but the ability to reach into annotated examples in the past and refer to them as it goes about a fix.

I’m seeing that frontier models will not follow intended patterns or know when to deviate if their primary charge is “fix this bug.”

It is perhaps a better idea to have agents’ primary job be “maintain this codebase” and one way that they do it is to fix bugs, and “oh by the way here is a bug.”

by codebje6 hours ago|

parent|

prev|

[-]

Before that we need human methods to determine what 'correct' should mean for our code.

(Performance can be part of 'correct', and can be just as hard to actually get right in specification!)