upvote
> ... Anthropic's Project Glasswing is supposed to find them quite a while ago?

That was my thought too. For all of Anthropic's talk about their "adversaries", it seems Z.AI have been quietly offering fixes for single shot Remote Code Execution flaws in US software (Safari / WebKit) that Apple and Glasswing / Mythos missed, and that Apple would not attribute to GLM.

reply
> and that Apple would not attribute to GLM

That was a wtf to me, so I checked Apple’s latest iOS release security content and GLM & z.ai is mentioned once (under WebKit), Anthropic is mentioned twice, Codex is mentioned once. Not clear if there are other instances where the model did most of the work but wasn’t credited. I didn’t bother to check other releases.

https://support.apple.com/en-us/128066

reply
> That was my thought too. For all of Anthropic's talk about their "adversaries"

It’s very likely they found all of them, but that the same happened that happened to Microsoft a couple of decades ago: NSA orders not to disclose / fix them so that they can put it in their collection of unfixed zero days.

reply
This is a coherent explanation for why federal model censorship has started with cyber capabilities. But this GLM model release is an in-your-face challenge to that policy. They now have to either set models free or impose a censorship regime that will put anyone not under it at an advantage. Or muddle along in the middle as usual.
reply
> Or muddle along in the middle as usual.

I'm not a gambling person, but if I was this would be my bet.

reply
Then "security through secrecy" is really bad mantra especially in the age of AI: others will find the same zero days very soon. If they attack you, then this loses the whole plot. If they propose a fix, then your arsenal becomes smaller.
reply
Who says they missed them? Could also be sitting pretty in CIA’s long list of ready to go Vault7-like exploits.
reply
deleted
reply
Probably Anthropic found them too and promptly got a call from Isreal to stop looking.
reply
> I understand the argument of "people are not actively looking", but isn't the cost for such a scan getting lower by the week, and Anthropic's Project Glasswing is supposed to find them quite a while ago?

You have to consider that having an LLM scan for vulnerabilities is hardly infallible. It is a search guided by heuristics and given a large enough codebase, it is unlikely to identify all vulnerabilities.

Personally, I've had Fable 5, GPT 5.6 Sol, and GLM 5.2 all looking for correctness issues in an old abandoned WIP codebase of mine and all of them found some that the others hadn't discovered. Now, correctness issues aren't the same as vulnerabilities, but the same principle about using heuristics to find defects applies.

reply
> [A]ll of them found some that the others hadn't discovered. Now, correctness issues aren't the same as vulnerabilities, but the same principle about using heuristics to find defects applies.

This makes perfect sense, but that conflicts with the impression put forward by Anthropic and OpenAI (in particular) that they alone occupy 'frontier model' spots. Frontier models should large dominate their competitors on a capability basis, but if GLM 5.2 (now 5.3) is routinely finding bugs / vulnerabilities missed by Fable and Sol then GLM might be genuinely a frontier-grade model by itself.

reply
> This makes perfect sense, but that conflicts with the impression put forward by Anthropic and OpenAI (in particular) that they alone occupy 'frontier model' spots.

Not necessarily. Even near the frontier, we don't really have a total ordering of capabilities, but a partial order. And even frontier models make plenty of mistakes. Combined with the randomness inherent in searching large codebases for vulnerabilities or correctness issues, it is entirely plausible that even much weaker models (and GLM-5.2 isn't even weak) can stumble upon issues that stronger models missed.

My current hypothesis – for which I have only limited evidence, unfortunately – is that it is better to have multiple reasonably powerful (but not necessarily frontier) models looking for issues than just one very powerful one. And even then you're likely to miss out on some issues.

reply
“Company hypes own product, downplays competitors” is still a thing with AI
reply
Fable 5 is just over two months old.

For normal software it would be as you say, but LLM progress is so ridiculously fast that things go from "bleeding edge" to "eh, you'll do" in about that timeframe, and "eh, you'll do" to "why even bother with this old rubbish?" in the same again.

Or, from a different perspective, we can expect some new frontier model from Anthropic in a week or two, and from OpenAI in a month or so.

reply
I find that LLMs also generate a lot of false positives, or extremely minor issues that don't warrant a fix (that are always overstated by the LLM as very important!). Signal to noise is still not great and requires somebody to wade through and pick out the actual good findings.
reply
> and Anthropic's Project Glasswing is supposed to find them quite a while ago?

We cannot trust a single company to report security issues, it’s good to see competition in that domain

reply
Open source competition no less.
reply
this is impressive and actually matches my expectations in terms of near term AI progress. we are going to continue to seem impressive progress in coding & related, anything where verifiability is scalable in an automated way: https://transitions.substack.com/p/a-quantum-of-ai-progress?...
reply
In a similar vein, does anyone know how to classify the kinds of problems that are being found?

Is it possible to build heavier traditional linting to catch whatever is being caught in a more deterministic way? It seems to me that would be far more efficient in the long run (even if the efficiency is only for the AI to know that aspect was already checked).

reply
> but isn't the cost for such a scan getting lower by the week

Not with Anthropic's models!

reply
Interesting... So Chinese models are not so bad?
reply
There's a chance that the real reason why they want to ban Chinese models is that they are so good at fixing bugs and preventing exploits that intelligence agencies have been using for espionage and surveillance for a long time.
reply
Anyone who knows anything realises banning things is a) impossible and b) your enemies will use them anyway, you are just depriving your own side of the advantages.
reply
Unfortunately, those in power, pretty much all over the world, lie/deceive themselves and believe they can.
reply
[dead]
reply
In this case depriving US companies would be the point though, so that's not necessarily a disadvantage.
reply
> Anyone who knows anything realises banning things is a) impossible and

Maybe "It's really hard" is more accurate? We (humanity) for most part basically agreed to ban the usage of various chemical weapons in wartime, which seems to have drastically reduced the usage of it, even though it's still used by shit actors today from time to time. But it's hard to deny that usage didn't decrease after banning it, which makes "banning" maybe not completely useless for certain things.

"Banning" things that can be easily copied over cyberweb transportation pipes feels like an fool's errand though, regardless of what it is. It's just too easy to get around, compared to actual physical items I suppose.

reply
This is different now. US labs and companies are not releasing frontier-level models openly (specially those capable of assisting cyber intelligence work), but commercializing them instead. Thus, any ban would not be symmetrical to begin with, and that is precisely what maintains the balance.
reply
deleted
reply
It's pretty easy for the US to functionally ban chinese models. They only have to target US firms like inference providers or the biggest users, and pretty much the whole domestic market will fall into line. They don't actually care about the final few %.

Regardless of whether or not adversaries are using them, the US has by far the most compute available, and we've now hit the line where major providers are no longer releasing their best models. The public gets the "current" level of intelligence, while the US government gets to control access to the actual frontier of non-public AI. From their perspective, their enemies using GLM5.3 while they have GPT6 and Mythos6 or whatever is a fine trade.

I don't support a ban at all, nor the US's behavior, I'm just pointing out some facts that change the argument.

reply
But the real bad guys will be this final few %, which defeats the purpose. The 99% will be average user which will swing to cheapest AI or easiest to access.
reply
I don't think this really works because the Chinese government is going to be incentivised to tip off the US companies to deny the US government those exploits. I guess maybe that's what the open source patch program here is about, making sure banning the models doesn't work because they can just report the exploits without the company running the model themselves.
reply
How does banning the models in the US prevent this?
reply
when in doubt, it's better to assume capitalism than anything else.
reply
Do you actually believe this?
reply
The CIA ran one of the world's largest cryptography companies, for DECADES[1]. Are you truly so naive that you believe intelligence agencies that have more to gain from stifling the discovery of vulnerabilities they know of and use wouldn't do so?

[1] https://www.washingtonpost.com/graphics/2020/world/national-...

reply
You should probably realise that the world has radically changed since then. This kind of thing works when you have a significant lead in the field that makes keeping vulnerabilities open sufficiently low risk for your own side. But if your adversaries have similar capabilities, then the calculation changes.
reply
Has anything changed? Governments are hoarding undisclosed vulnerabilities, using them as they see fit instead of fixing. Every espionage, surveillance, or war campaign (see Russia v Ukraine, US/Israel v Iran etc) is followed by a ton of burned 0-days.

>This kind of thing works when you have a significant lead in the field

No? It works even if the adversary has the same capabilities. It only stops working when everything is fixed.

reply
I believe it is unlikely. (Not because I do not believe NSA is hoarding 0-days, but for many other reasons.)

I'm curious: to any professional vulnerability researchers reading this, what do you think?

reply
I used to call everything a conspiracy theory, but then Glenn Greenwald published "No Place to Hide: Edward Snowden, the NSA and the Surveillance State".

Now i know that reality is worse than the worst conspiracy theorist.

reply
I don't think reasonable people post here much anymore. It's mostly galaxy brained conspiracy theorists and ignormamuses posting political garbage. Reddit-lite on the way to full blown Reddit
reply
Why would you even believe the opposite? US spooks have been amassing vulnerabilities and relying on them for decades, they literally pioneered it in the 90's if not earlier. Everyone does it now but the US is the biggest of them all. Surely this devalues a lot of what they did. Moreover, the way the US government handled new capabilities, and OpenAI's training policy (they are in bed with the government) just scream "we want to create weapons for cyber-offence and deny them to everyone else"

It might not be the reason, but of course it's a contributing factor.

reply
[flagged]
reply
Good thing I said nothing of that (especially nothing about China). Reread it again to understand you built an incredible strawman and ignored my last sentence.
reply
Well we know that the US government is pushing to restrict access to such models while the Chinese are publishing them for free, so it's mostly a matter of motivations, not the actual facts of the matter. And the USG has a documented history of unsavory behavior (including toward its own citizenry) in that area.

So we might ask if one of the reasons the US is being the bad guy is it's usual spying antics, and we're left asking why China is being the good guy.

reply
Intelligence agencies have been known for exploiting and planting software and hardware Buga for decades, going as far as weakening cryptographic standards or intercepting hardware in transit to implant a backdoor device.

Why do you _not_ believe it's a possibility?

reply
Critical thinking says this is not only possible but likely too.
reply
They've always been good enough for double digit less money. Always. Anyone thinking "Chinese models fake models built using dirty distillation scam" don't know what they're talking about.

Distillation is just forcing the model to use an exam prep workbook for training instead of generic publicly available textbooks. The models themselves has to be smart enough for that to work. It's the exact same thing as Asian tiger mom double schoolwork strategy, to paint a picture.

reply
To carry on this analogy - do test prep workbooks make you meaningfully more competent in general, or is it benchmaxing? (Versus studying textbooks for a similar time, of course.)
reply
Their best models are getting more and more expensive, and still aren't SOTA.

It's almost like there's an actual cost to developing these models, and the Chinese don't have magic dirt that allows them to do it at a fraction of the cost.

reply
Looks like they're going for good PR now, to avoid smearing by the "Western" models. Smart!
reply
I'd love to live in a society where people and corporations do good things for PR.
reply
> I'd love to live in a society where people and corporations do good things for PR.

Maybe so, but I'm not sure I'd like to live in China of all places. (Don't get me wrong. Lotta places I'd like to visit if I ever got the chance, and China's on that list, but to live there? I don't think so.) Maybe one of the Nordic countries?

reply
PR for good things doesn’t make money.
reply
If you look at the distribution of their findings in the linked post, most of theirs are issues introduced a long time ago, almost all before 2006.

Complete speculation, but I wonder if they and Anthropic are scanning very different codebases and Anthropic's skew would be in the other direction.

reply
amazing! huge clusters in code from the 1980s haha
reply
> Anthropic's Project Glasswing is supposed to find them quite a while ago?

Someone still has to run it. The analysis and fix could be someone's machine but not committed / published.

reply
Wordpress having a high number of vulnerabilities not surprising lol
reply