The real question is will [when will] these vulnerabilities get fixed faster than they can be found. LLMs are finding a lot of real bugs that have existed without being noticed for 20+ years. It is generally expected [hoped] that in a short time all those 20 year old bugs will be fixed and LLMs will settle down to just things introduced in the last month (or perhaps this pull request) and thus be manageable.
replyWon't an LLM, prompted for a CVE report, always hallucinate one even if it isn't there? Why would they "settle down"?
replyThat's becoming less common. The issue is that, if training only contains examples of CVE reports, then asking for one will get you one, even if it's not grounded in reality. But if training consists of many negative cases with the response, "I can't find one," them that will become the default unless evidence pushes it towards a report. It's a classic class size problem. SOTA models seem to have this largely solved, though a determined person (just like a determined boss) can prompt one into giving a false answer just to satisfy the request.
replyTwo different points.
replyFirst, is there any reason they have to? The LLM makers are aware of this problem and doing things to stop it. Maybe they will start saying "I can't find anything". Already they refuse to provide some exploits.
Second, most people are not prompting for a CVE, they are asking to review and find issues if any. Which sometimes finds a CVE, but in my experience it sometimes does say "looks good".
Hence future CVEs on a lot of projects will likely demand working exploit code for testing. This at least attempts to lock the issue down to something more tangible.
reply