upvote
Letting Claude get at the source code to try to find CVEs. I found it particularly entertaining that after finding none it just devolved to a grep for "strcat."
reply
Oh, I see. No, you're wrong. That's absolutely not what it did and not at all an accurate way to sum up what it found.

This isn't a complete rebuttal to your argument but I'll note with irony that we're commenting on a thread about a FreeBSD kernel remote that Claude both found and wrote a reliable exploit for (though people will come out of the woodwork to say that reliable exploitation of FreeBSD kernel remotes isn't much of a flex).

Here, from the exact tranche of vulnerabilities you're saying was just a "grep for strcat", are the Firefox findings:

https://www.mozilla.org/en-US/security/advisories/mfsa2026-1...

We're getting to a point, like we did with coding agents last year, where you can just say "I believe my lying eyes". Check out a repository and do Carlini's "foreach FILE in $(sourcefiles); <run claude -p and just ask for zero days starting from that file>". I did last night, and my current dilemma is how obligated I am to report findings.

reply
It's from the link I posted. Claude's own team in January trying to do exactly what you suggested and ending with results that are less than promising. It's their blog. I assumed it represented the pinnacle of their research.

We're getting a point where anecdotes are being used in place of reason. I'd think you want to ask "how many bug bounties are earned by humans vs AI assistants?" If there's money to be made in finding 0-days then shouldn't there be ample evidence of this?

reply
You can see now that you assumed wrong.
reply