I see this as a fault with Meta's models, not a fault with the concept of open weights. The Llama family just aren't very useful. They make flowery prose but they're terrible at tool calling [1][2][3] so there just isn't much I can actually accomplish with them.
[1] https://gorilla.cs.berkeley.edu/leaderboard.html [2] https://benchlm.ai/best/tool-use [3] https://benchlm.ai/llm-agent-benchmarks
At the time the whole thing was led by Yann LeCun who seemed to spend more time arguing with people on Twitter than figuring out new techniques to make Llama the best. Meanwhile Deepseek was figuring out large scale RL on kneecapped hardware like H800s and how to scale architectures an order of magnitude bigger with MoE.
Meanwhile Chinese labs were forced to innovate with more efficient models.
He was not in charge of the Meta LLM stuff, from anything i've read over the past few years.
Which brings me to my other point. If you want to compete with serious labs (OpenAI, Anthropic, Deepseek, Alibaba) you need to be smart and focused, not messing around.
They care about control. I see many of my enterprise (or just-below-enterprise) clients very annoyed at OpenAI and Google after 2-3 years of model toil, where they had to constantly re-calibrate onto new models, on tight externally mandated deadlines, with little certainity. Now they are reaching for open weight models instead, that they currently host with the same inference providers, but have the option to in-house if push comes to shove.
In this context, open source/self-host means do it yourself, closed source means you trust someone else to do it for you.
In the long run open source always wins because of the community effort, customization, network effects and price.
Internet protocols are open, anyone can setup a website, host their own email server..., but most people don't do that, they rely on someone else to do it for them.
People still pay for Windows rather than use Linux because most people and companies have better things to do.
The main threat to American AI companies is not that consumers will self-host, but it's that new hosting companies will appear that will host open source models and offer them (cheaper) to consumers. It'll be like the web hosting market before the era of cloud computing.
I don't think that's what it means in this context. Hardly any users are training their own models, after all. And I don't think Windows/Linux is a good analogy - OSes have inherent platform lockin that LLMs don't.
Really the only 3 things anyone cares about are capability, cost and data privacy. Sure, the cost axis for open weight models needs to include the cost for hosting things yourself, but I think the bigger reason that businesses have been throwing billions at Anthropic's and OpenAI's models is that they have had the best models, and they've had big releases every few months. Their biggest Achilles heel is that if/when their improvements start to plateau, open models may catch up and businesses will start to scrutinize their AI spend a lot more.
To me, Llama was the ONLY successful thing they did it. it was when they stopped that they fell off the radar as an interesting AI company. They had a genuine chance to be the "substrate" that Ben talks about here. I can't actually figure out why they threw it away.
This resulted almost every time in "screw off we'll train our own models or use refined open source ones instead" leading to a lot of anger at Anthropic by CEOs these days.
We all want an alternative and Anthropic and OpenAI need to charge more than they are worth to pay back their investors and everyones stuck now.
Edit: here’s the zitron post at the time: https://www.wheresyoured.at/anthropic-is-bleeding-out/
The llama drama will be a netflix show of it's own in 5 years.
This is also a strange way of framing it, though. Llama was released as a research project, it was never intended to create some vast ARR revenue stream or reframe the way people look at AI. If Meta wanted to exploit it for personal success then they had lots of opportunities to do so.
With OpenAI and Anthropic's profitability under question, it is up in the air whether or not America's stance towards AI will work. If they can't convince the world that they're a proper software business, then China's philosophy will win by-default.
At the moment, I can't say that I see this happening. It's hard to know whether it may happen in the future.
From what I see it seems like we're hitting the top of a sigmoid curve in the model's utility for coding assistants. Going from "it does 90% of the job" to "it does 94%" of the job is a legitimate improvement, but it's not a phase change, and it's probably not worth paying multiples more for. And coding assistants have turned out to be the killer app for AI; it still isn't really working out in a lot of the rest of the industries of the world.
At any moment, theoretically, someone could find some new way of making AIs that breaks this sigmoid and propels us into a new one. But that's not a great thing to bet the farm on.
I'm not sure I'd say "China" wins if this particular strand of American AI fails. Falling back to an open weights model and making money on the serving of the models wouldn't take all that much economic realignment for the US and would be the natural outcome of any sort of fire sale of current AI assets. However the devastating effect on the stock market if the market comes to the conclusion that this current round of AI can't be profitable without falling back to such an economic posture and some more years of people adjusting to it can hardly be overstated.
They release the base model as open source, everyone uses it. They make a paid version, no one uses it.
They make no money from the open source version, they get no social credit from it. Where is the benefit to having an open source model?
Research. Llama is and was a research project, intended for researchers. You could make this same critique of Microsoft's Phi model, Apple's OpenELM or OpenAI's OSS. None of them were intended to be kingkillers, all of them are experimental in nature.
You might not have followed the space at the time, but there was a real race to implement the transformer architecture with fewer overall parameters than GPT-2 and GPT-3. Llama was revolutionary for sticking the landing without being entirely lobotomized, the "benefit" was that the model was usable on a local machine. Contemporary projects like Flan-T5 and GPT-J/GPT-Neo were entirely displaced, Meta's AI mindshare went ballistic for a few months and probably propped up billions in exit liquidity for executives and former employees.
The open source character of the models is irrelevant if you need a nuclear powered data center for inference. In the end it is just another internet service.
That's true.
> So, the ‘better’ Chinese AI gets , the more it will just be a service run on Chinese hardware competing with ‘our’ lower latency AIs .
This is not. Them being open models means that any hosting provider in the world can host them as well. You get to pick and choose the provider the same way you'd pick and choose where to run a Linux server.
Cost of inference is small when compared to the cost of training new models. Which is the real advantage these Chinese models have. Someone else has already spent the capital needed to create the model.
With a reasonable upfront investment and a few trained staff, its very possible to run these larger Chinese models in a well managed fashion. The real calculus is if this up-front investment and associated lifecycle costs are over or under the costs a corporation may simply wish to dump into a cloud managed service like OpenAI.