The "if" is doing a lot of heavy lifting. Can it do what it claims to do? LLMs are always hallucinating that they've found the most severe bug in software history.
reply> LLMs are always hallucinating that they've found the most severe bug in software history.
replyThat is a function of the training data/method, like many of AI's problems. This doesn't stop it being a problem of course, but it shouldn't be a surprise.
Big bugs, like heartbleed, over the years have had breathy websites going over how serious the problem is because the problem was that serious, so the models associate that sort of wording with bug reports. Less serious bugs don't get the same excited treatment, repeated in many many places, so more measured descriptions figure in the training data relatively less.
Good point, I don't know! That's why I formulated it this way.
replymind boggles, issue is they're swamped with AI slop and high number of findings, they're using AI assistance operated by a 3rd party to handle researcher submissions so many legitimate findings slip through the cracks.
replyThat's unfortunate. They need to do some kind of triage then, I hope this doesn't mean that they're constantly busy now with equally bad or worse bugs, but I guess it wouldn't be too surprising.
replyThey triaged them - that's why it gets wontfix.
replyFalse negative? I honestly don't know what we can do about it. If this is a real issue, maybe another llm will rediscover it and it eventually will get triaged right?
Ironic comment from someone who literally just submitted an AI slop report and went so far as to give it a website and cutsey name to attempt to feign legitimacy and skill behind the "researcher."
reply