The memory manufacturers have been through bubbles before.
I don't deny that AI is "real", in that there is certainly something there, but I would hate to be the guy betting billions or even trillions of dollars that we're not in a demand bubble well in excess of what is justified by that tech at the moment and expanding capacity is a great idea. Of course, I would also hate to be the guy saying "no we shouldn't expand capacity" when it turns out that, yes, we should have, but the staggering profits being made in the meantime would cushion that blow pretty well.
That's true. But will anyone have any money to pay for the LLMs? The AI market is currently being heavily subsidised by, ultimately, everyone else. But the economy in general is looking extremely dire at the moment, and that seems unlikely to last.
Can you elaborate on this? What about the economy is looking dire?
And from what I read in the media, a lot of the economic indicators are backing this up: discretionary spending is down, private debt levels are rising, etc. All signs that the slack in the system is disappearing.
From the bottom of my heart, I hope that's the future.
I'm not convinced that will be the case. The AI companies don't want you to have local control. They want you to subscribe to a service that they can change at any time.
I grit my teeth when I type this, but (god help us) I think Apple is maybe the best (least bad?) hope here. They are the only big player with the hardware chops and without a current vested interest in getting you addicted to monthly AI subscriptions. I'm not saying this is highly likely... I'm just saying that out of the current major players, they're the ones with the ability and motivation to move in this direction in the near future.
The next best hope is probably just nVidia or AMD catering to the consumer market once again, after the AI bubble bursts or the datacenter market reaches saturation.
The outside hope is that some startup makes a business out of burning open weight LLMs to silicon like ChatJimmy... although ChatJimmy was bought up by AMD AFAIK.
The relevant questions are: 1) Where are the economies of scale in the technology stack? 2) What's "good enough" to consumers, and how does that stack up with the relevant computing power needed? 3) What are the transaction costs along various system boundaries?
I think that the biggest force keeping inference in large centralized services is simply that provides a better product for the average user who doesn't care about local control (and the average user doesn't care about local control; indeed, for most people it's a misfeature, as then they have to administer their own hardware). HN is full of nerds that want to own their own stack; they're willing to put up with a little loss of capability to run Qwen 3.8 locally. But from the folks I know that have tried it vs. Claude vs. Codex vs. Antigravity, the local models are still pretty weak compared to what you can get by paying for a service. Most people will just pay for the service until the performance becomes indistinguishable and the price becomes less.