- It is trivial to extract samples of the training data that was used, which can bolster existing lawsuits/foster new ones.
- Older models are not as safety-hardened, so it is easier to coax unsafe behaviour out of them, which is a PR risk.
- It may be possible to divulge proprietary secrets from the model (e.g. architectural details that may still be relevant).
For these reasons, and more, it's unlikely that GPT-3/similar models will be released until these concerns are no longer relevant (e.g. when they become a purely historic concern, similar to the open-sourcing of other proprietary software from decades ago).
At least to this point, the legal teams could get the model via discovery. IDK that the lawfirms realize that they could get experts (or that they'd have contacts that could) to search the model weights.
The weights arent the truth tho, maybe a timecapsule-vhs but i wouldnt trust llm weights more than more hardcore deterministic media that might get preserved to infer facts from an era.
The companies doing the training are becoming the "winners" that are "rewriting history" as they train their models.
According to the book The Thinking Game, lower quality models at that time were considered less safe, because they could be easily tricked into doing harmful stuff. In the book, Dario (of Anthropic) was the head of safety at openAI and was responsible for pushing for 10x scaling in training to make the models safer.
It does make sense, a smart model is going to be way harder to jailbreak into telling me how to synthesize anthrax (or something).
In the times of GPT-3 I'd scoff at the idea of an LLM doing any hacking; today, I'm running several AIs on my code before publishing, and they are finding (and demonstrating!) RCEs on my localhost server.
For example, one found a missing check in a third party JWT library which allowed full account takeover, which I'd have never even looked at.
Hence I don't believe a single word coming out of these people's mouths. Their "beliefs" are just marketing.
I certainly think the labs have muddied the waters using safety for marketing, but that doesn't mean less capable models weren't more dangerous at one point.
Easy access to malicious information hasn't proven to be the disaster these anti-prophets proclaim. For the last ~3 decades of widespread internet and search engines, you could trivially find all sorts of information (drug synthesis, explosives, etc.), and we're just fine.
(Corollary: easy access to good information did not make non-intellectual non-curious people smarter. Easy access to information does not have the consequences people immediately assume.)
My guess is I’m unintentionally refusing implicit signal that you think it’s safe to release all models openly, because you have observed models finding a vulnerability in a JWT library. But that sounds like a straw man instead of a steel man. Idk. :/
According to me, this is nonsense.
What's the point of publishing it when it'll likely be outclassed by gpt-oss?
AI policy is being shaped somewhat by the things Sam and Dario say. So even if you're not feeling vindictive, it's probably good to keep a track record of the previous things they have said as a Bayesian prior. People who don't know better listen to these people, and maybe they shouldn't.
I think the release of kimi k3 is definitely arguably dangerous, we're already seeing consequences of elite-tier cyberoffense capabilities.
I cannot stand using gpt-oss, but I miss some of the creative spark of GPT-3 davinci dearly.
Or did you just mean the weights? If so, the open models distilled from GPT3 should mostly suffice. That's what good "open weights" are.