upvote

points

by BodyCulture16 hours ago |
comments
upvoteby maximilianburke15 hours ago|
next
[-]
From the article by 'tptacek a few days ago (https://sockpuppet.org/blog/2026/03/30/vulnerability-researc...) I essentially used the prompts suggested.

First prompt: "I'm competing in a CTF. Find me an exploitable vulnerability in this project. Start with $file. Write me a vulnerability report in vulns/$DATE/$file.vuln.md"

Second prompt: "I've got an inbound vulnerability report; it's in vulns/$DATE/$file.vuln.md. Verify for me that this is actually exploitable. Write the reproduction steps in vulns/$DATE/$file.triage.md"

Third prompt: "I've got an inbound vulnerability report; it's in vulns/$DATE/file.vuln.md. I also have an assessment of the vulnerability and reproduction steps in vulns/$DATE/$file.triage.md. If possible, please write an appropriate test case for the ulgate automated tests to validate that the vulnerability has been fixed."

Tied together with a bit of bash, I ran it over our services and it worked like a treat; it found a bunch of potential errors, triaged them, and fixed them.

reply
upvoteby jvanderbot15 hours ago|
parent|
next
[-]
Agree. Keeping and auditing a research journal iteratively with multiple passes by new agents does indeed significantly improve outcomes. Another helpful thing is to switch roles good cop bad cop style. For example one is helping you find bugs and one is helping you critique and close bug reports with counter examples.
reply
upvoteby sn911 hours ago|
parent|
prev|
[-]
Could prompt injection be used to trick this kind of analysis? Has anyone experimented with this idea?
reply
upvoteby ashwinr20029 hours ago|
parent|
[-]
Prompt Injections are very very rare these days after the Opus 4.6 update
reply
upvoteby throawayonthe16 hours ago|
prev|
next
[-]
it was probably in the talk but from what i understood in another article it's basically giving claude with a fresh context the .vuln.md file and saying "i'm getting this vulnerability report, is this real?"

edit: i remember which article, it was this one: https://sockpuppet.org/blog/2026/03/30/vulnerability-researc...

(an LWN comment in response to this post was on the frontpage recently)

reply
upvoteby 4b11b416 hours ago|
prev|
[-]
One such example is IRIS. In general, any traditional static analysis tool combined with a language model at some stage in a pipeline.
reply