upvote
The ram/gpu shortage won't last forever though. Moreover we can be pretty confident that long-term the prices will obey wrights law and come down in cost significantly (from the pre-shortage prices) as we learn to produce them more efficiently.

LLM companies are valued as if they're going to have some enduring monopoly that they can extract money from... GLM-5.2 and similar models make that valuation very very questionable.

reply
> The ram/gpu shortage won't last forever though.

No disagreement there, but it could easily last another 3 to 5 years which is a long time in tech terms.

reply
Long enough for them to IPO and all the execs to retire. I doubt they care beyond the IPO.
reply
I think this is the play
reply
> The ram/gpu shortage won't last forever though

Don't underestimate the markets ability to remain irrational

reply
the companies which have the power to alleviate these shortages are the same companies who are profiting most from the shortage. scarcity is an asset, it's not irrational that a concentrated marked will produce more of that asset.
reply
The solution for high prices is high prices.

If making RAM and SSDs is now cause for a 10 figure valuation, after enough time somebody will dive in.

reply
What's the irrational part? There's sky high demand.
reply
maybe the irrational part is the amount of demand for consumer hardware, wouldn't the market for professional ML/AI used hardware go away from consumer hardware over time? (I can talk more about what I mean consumer hardware to be)

Also irrational parts of this market (would love to hear your thoughts):

- the purchase of hardware that isn't power efficient or gives an ROI for ML/AI use cases by companies buying it, who would be priced out of using that hardware over time

- many people and companies are buying the hardware due to hype and scarcity/FOMO reasons over rational reasons

reply
Very few people, but quite a lot of companies especially after per token pricing took effect and companies see their invoices skyrocketing. You pay an upfront cost once and you’re done.
reply
is it possible that ai companies ordered a bunch of ram just so that models cannot be run locally? they are betting new fabs wont be built before quantum takes hold.
reply
When a large open weight model is released, a lab, startup, or a rich hoist can easily do logit-level distillation and create a XXb param model or whatever, and in theory you should get a really good distill.
reply
I suspect the time horizon is shorter because of software advances. We are getting more capability out of smaller models

Alibaba released Qwen 3.6 "tiny" models not that long ago, they punch way above their weight(s)

reply
> Alibaba released Qwen 3.6 "tiny" models not that long ago, they punch way above their weight(s)

True, Qwen3.6-27B is amazing for it's size. However, it seems likely that we're not going to see anymore of these smaller models from Alibaba/Qwen since several key players exited that organization a few months back.

reply
Do we know where those key players went?
reply
Good to know, I think the trend is clear based on the models coming out of China and well see more capabilities in smaller, more efficient models.
reply