Now what?
The fundamental problem here is incentives and tactics. Either the models are actually better (which I think the iPhone to cheap Android phone really speaks to, i.e. they do the same thing but one is 50x better at 5x-10x the cost) and thus they can be gate kept and like the iPhone the vast majority of profits go to a select few with high end implementations. OR the models aren't actually that much better, companies lose a fortune and then nobody can create any better commercial models or build out scale needed for open source models because it's not profitable.
We could wind up with only open-source models or something along those lines, but if the compute and scale is needed to train the models, nobody will be able to do that profitably and so AI research is either gate kept and silo'd for something like military applications or it just doesn't really happen because there's no funding for this scale of build out.
Try mmapping > 5GB file in your 50x better iPhone.
Try running any service in the background.
The list goes on and on.
Your 50x better suddenly became 50x worse compared to a much cheaper android.
androids and iphones are approximately the same thing
the kinda obvious direction LLM training can go is into the direction of particle physics, and the training is set up democratically and through universities and via multi-state funding
then the resulting weights end up open, the same as the particle detection data
Yet...
Android holds 70.6% of global active devices to iOS at 28.7%, but iOS captures 64.2% of consumer app spend. [1]
Possible, certainly. But this case also applies to China and its "open-weights" strategy. They won't be able to form companies either or get ahead.
[1] https://www.digitalapplied.com/blog/mobile-os-market-share-2...
Based on the above stat, it sure seems like Androids more versatile and inexpensive for a much larger group of users.
Hmm...that sounds familiar....
Why do you assume the poor schools wouldn't be smart enough to keep it going? It's very likely the can collectively beat the rich school now that the one other rich school opened access to their materials and led the charge.
> but if the compute and scale is needed to train the models, nobody will be able to do that profitably
But they would. Efficiently hosting models will be the real business and early access to models with incremental improvements will not be the moat once thought. The reason other companies don't feel they can compete is the same reason OAI and Anthropic will lose their lead. They banked too heavily on another player NOT leading the charge on open research and poured disgusting amounts of money at closed source models.
China has proved they can take the limited resources available to them and build something better than what the US is offering consumers [1]. I'm just waiting for other countries to start pitching in.
Reminds me of the NSA and their early battles with cryptographers who believed in open research.