Is there a VirusTotal.com-but-LLM-analysis that folks could link to instead where we'd trust the prompts were sent and the responses were indeed received from the stated models? Hopefully run by someone with quite the budget and/or reputation.
This is how people learn.
I think saying that it contributes nothing because a) someone could do it themselves, b) the output might be slop, and/or c) they could be lying, is a bit silly. Those things apply to basically everything posted on the internet.
Whether an LLM security review is actually valuable is an entirely different discussion.
+1 at least he cited his sources lol
Just toss GBs of file structure: "AI, do your work baby!"
I for one break things down much smaller into very specific tasks involving very particular text. Maybe I'm overdoing it lol.
For me, an AI security review would still take hours or days, it would hardly be a 1-shot prompt like this.
But it quickly loses fidelity as you load more into the context. The context window is supposed to be much larger, but in reality, it loses accuracy and fidelity the more you load in.
If I loaded 10k+ lines of code across files into a RAG db (since that's much too large for LLM context) - which is what the foundation of "an agent" is - I highly doubt that it would be very effective on its own. And it isn't IME, that's why so-called agentic coding isn't very good compared to an expert using an LLM manually, breaking it down into task-specific work.