upvote
Some combination of reporting bias given concerns about LLM security capabilities and actual new vulnerabilities found with LLM assistance. Even if exploits and outages are unrelated to LLMs, I'm certainly thinking about whether claude could build these things (or if actors already have).
reply
> What is happening?

Slowly at first, and then suddenly. AI assisted anything follows this trend. As capabilities improve, new avenues become "good enough" to automate. Today is security.

reply
i believe a good portion of the cves hitting the front page are moreso because they are ai-related (found partially/in whole by ai) and make for quick upvotes.
reply
AI is happening.
reply
In each recent case?
reply
AI assistance was explicitly disclosed on yesterday's. Today's has Claude as one of two contributors on this GitHub Pages site at least so it's also very likely.

Agents are capable of finding this kind of stuff now and people are having a field day using them to find high-profile CVEs for fun or profit.

reply
In some sense, I wonder if non-open-source is "safer" since LLMs can't mass scan the code for exploits.
reply
Maybe for a while, but there's nothing stopping LLMs from examining disassembler output.
reply
Security through obscurity
reply
If they don't get scanned, then they also don't get fixed, so if they have the same amount of holes, they will stay vulnerable for longer.
reply
It's actually the perfect evergreen content to discuss on HN in an age where so much else is AI generated.
reply
Automated vulnerability discovery via LLM.
reply
Anyone care to share which models and which prompts actually lead to finding these kinds of vulnerabilities? Or the narrowing-down workflow that can get an LLM to discover them? Surely just telling claude "Find all vulnerabilities in this project LOL" isn't enough? I hope?
reply
The Anthropic researchers have said their flow is as simple as:

1. Pick a file to seed as a starting place.

2. Ask the LLM (in an agent harness) to find a vulnerability by starting there.

3. If it claims to have found something, ask another one to create an exploit/verify it/prove it or whatever.

4. If both conclude there is a vuln, then with the latest models you almost certainly found something real.

Just run it against every file in a repo, or select a subset, or have an LLM select files with a simple "what X files look likely to have vulns?".

So basically yes, it is that simple. It's just a matter of having the money to pay for the tokens.

reply
Thanks for the reply. Pretty remarkable.
reply
Everyone was talking about how Mythos was overblown marketing, and while it may be, they missed the forest for the trees. Capabilities have been escalating for a year now and we're at the point of widespread impact. I don't suspect we'll see a slowdown for a long time.
reply
I agree. It is not like Mythos or other LLMs are insanely smart/superhuman. Many of these vulnerabilities could be discovered fairly easily by trained human experts as well. The problem is more that it requires an insane amount of attention and time of highly-paid experts to shake out these issues vs. an LLM that never gets tired and can analyze a large amount of code at low cost.

Linus' law was wrong because there were never enough (qualified) eyeballs to check the code. LLMs provide an ample supply of eyeballs (though it's not a benefit to open source, since proprietary developers can use the same LLMs).

reply
Same applies to them being good enough to program, but many are so focused on source code generation that they don't get the whole picture.

Thanks to agents and tool calling, there are now business cases that can be fully described by AI tooling, the next step in microservices, serverless and what not.

Naturally with a much smaller team than what was required previously.

reply
A mix of AI and hybrid warfare.
reply
Perhaps it was the prior quiescent period that was the anomaly.
reply
I wonder where are the Rust naysayers hiding now

C code is broken - period

reply
... there's also a bit of a frequency illusion factor.
reply