To be honest this is an even bigger problem with Semgrep and other SAST tools. Developers just want the .1% of findings that actually lead to issues, but flagging patterns will always lead to huge false positive rates.
I do something similar as what you suggested and it does work well -pattern match + LLM. The downside is this only applies to SAST and so far nobody has found a way to address the findings that make up 90% of a security team's noise, namely SCA and container images.
I assumed I was still dealing with lots of false positives from Gemini due to using the free version and not being able to have it memorize the full code base. Either way combining those two tools makes the review process a lot more enjoyable.
100% agree - I spun out an internal tool I've been using to close the loop with website audits (more focus on website sec + perf + seo etc. rather than appsec) in agents and the results so far have been remarkable:
Human written rules with an agent step that dynamically updates config to squash false positives (with verification) and find issues while also allowing the llm to reason.