> The llms know about every previous disclosed security vulnerability class and can use that to pattern match

Do the reports include patterns that could be matched against decompiled code, though? As easily as they would against proper source? I find it a bit hard to believe.

Many vulnerabilities aren't just pattern matching though; deep understanding of the context in the particular codebase is also needed. And a novel codebase means more attention than usual will be spent grepping and keeping the context in focus. Which will make it easier to miss certain things, than if enough of the context was already encoded in the model weights.

Same thing applies to humans: the better someone knows a codebase, the better they will be at resolving issues, etc.