Industries always consolidate and winners emerge. SOTA LLMs look like a natural monopoly or duopoly to me because the cost to train the next model keeps going up such that it won't make sense for 20 competitors to compete at the very high end.
TSMC is a perfect example of this. Fab costs double every 4 years (Rock’s Law). It's almost impossible to compete against TSMC because no one has the customer base to generate enough revenue to build the next generation of fabs - except those who are propped up by governments such as Intel and Rapidus. Samsung is basically the SK government.
I don’t see how companies can catch OpenAI or Anthropic without the strong revenue growth.
It's believable that Meta, ByteDance, etc. can catch up too. It is not certain that scaling will meaningfully increase performance indefinitely, and if it stops soon, they surely will. Furthermore, other market conditions (US political instability) can enable even more labs, like Mistral, to serve as compelling alternatives.
Uber, TSMC, etc. have strong moats in the form of physical goods and factories. LLMs have nothing even remotely comparable. The main moat is in knowledge, which is easy to transfer between labs. Do you think all the money that goes into training a model goes into the actual final training run? No, it is mostly experiments and failed ideas, which do not have to be repeated by future labs and offshoots.
It's certain that it won't. We've already hit diminishing returns.
I’ll be polite and call this statement ‘a very debatable’ one.
Only one company on Earth can make the UV lithography machines TSMC buys for their highest end fabs, and they're not selling to anyone else.
The PRC tried to brute force this supply chain backed by the full might of the Party's blank check, all red tape cut, literally the best possible duplication scenario, and they failed.
no, most industries just sell boring generic products, a few industries favor monopolists. Semiconductors are one of them but LLMs are also as far removed from that business as is physically possible.
TSMC makes the most complicated machines humans have ever built, a LLM requires a few dozen nerds, a power plant, a few thousand lines of python and chips. That's why if you're Elon Musk you could buy all of the above and train yourself an LLM in a month.
LLMs are comically simple pieces of software, they're just big. But anyone with a billion dollars can have one, they're all going to be commoditized and free in due time, like search. Copying a lithography machine is difficult, copying software is easy. that's why Google burrowed itself into email, and browsers, and your phone's OS. Problem for openai is they don't have any of that, there's already half a dozen companies that, for 99% of people, do what they do.