upvote
Is it a given that it's as good at outputting secure code as it finding insecure code? That leaves a fair amount of room for people not baking in deep review steps. Also mentioned elsewhere, but if velocity is way up...that ramps up bugs anyway.
reply
No, it's not a given. AI always finds security issues in the things I just had it write.
reply
But that should be part of your pipeline, no? Personally, the thing I've spent the most time working on with AI is.. a stack to get AI to keep a project on track and follow strict code writing standards that are reviewed. Not everyone is working in that, but I'm sure any serious development will be done via something similar.

Although, at the pace of improvement, I'd bet it won't even be necessary in a year or so.

reply
This is true for human written code too. This is why in the future all written code MUST be run through an internal security sweep before being given to your harness for writing. That's just basic due diligence at this point.
reply
As long as model providers deny access to security-capable models, it can't be basic diligence.
reply
This is something agents already do as part of their normal work. Providing code that you know is vulnerable is in fact a dereliction of duty and negligence. Don't provide a POC. Simply that as it produces code a background agent just checks it without showing the work and points out potential problems.
reply
The slop is cheap but checking will cost extra, what a pricing scheme! Modern day Pharo level of lockin!
reply
Yes but LLMs famously produce vastly more code than humans. My hot take: LLMs will produce more insecure code than they can catch, and will also find ways to smuggle insecure code to avoid extra work.
reply
Do we have any evidence around the rate that AI emits security bugs vs humans previously? You would think that perhaps AI would be better about this, but perhaps not.
reply
I'm making no claims. I'm just saying that contributing before ai was difficult so people rarely did it. Now that contributing is easy, more code will be added. More code = more bugs. Always.
reply
Yet the opposite seems to be true. Many projects now shun new contributors due to AI.
reply
Eh I'm not sure that's an indicator that the opposite is true. Github's vastly increased commit activity is the best indicator we have of this. https://www.tomshardware.com/software/github-report-highligh...

Sure, some projects are trying to be responsible or are trying to keep waves of slop PR's from drowning them, but that is by no means an indication of a general distaste for AI code. It's been largely embraced.

reply
Are any of those large projects used by many people?
reply
I've run Daybreak against my human written pre-AI code and found so much stuff that I don't trust human or AI written code. It all MUST be audited by a security sweep AI.
reply
Even if Fable and Astra are superhuman at identifying and fixing bugs that does nothing when projects get bugslopped faster and cheaper by some low-end model.
reply
I hate these meta conversations on crunchy technical topics and feel like we should be digging into the KVM security model more than this general security prognostication stuff. Having said that, I will hypocritically further this debate by pointing out that the determinant of software security moving forward is not so whether AI is drastically better than humans at avoiding common vulnerability patterns, (or instead just a little better). Rather, it's that AI will make it much easier to move subsystems or whole projects to memory-safe programming languages and towards formal verification.
reply
Existing code will have security holes that haven't been found yet.
reply