So an LLM was unable to reason about a codebase to find cross-library vulnerabilities.
replyYour response was a weak excuse, it’s a clear demonstration of the shortcomings of LLMs which will inevitably cause headlines in the future.
If you point an LLM at a middleware and ask it to find vulnerabilities, then not finding this is a shortcoming.
replyWhether "LLM failed to spot vulnerability that took humans 8 years to find" is a great headline about shortcomings of LLMs is questionable, but it is a good example of a category of bug that is particularly hard to spot for humans and LLMs alike
When the past month has been full of headlines claiming that Mythos et al. will be the end of secure software as well know it, it's fair game to emphasize the places we know already are not going to be covered by them.
reply