Even if the bubble pops and anthropic and openai et al implode - genie doesn’t go back in the bottle. The usefulness of LLMs for coding is proven, and a chip in a datacenter running 24/7 is always going to be more valuable than in a personal device running occasionally.

That doesn’t change until production capacity exceeds the datacenter demand. When that happens, they’ll start selling them down the market until it eventually reaches phones and toasters and whatever. But not in two years.

> * Software inference optimizations

Absolutely. I'd be surprised if they couldn't 2x performance in the next year. Still doesn't make a 1T model fit on your phone.

> * Heavy quantization

I think this is a dead end if you're trying to fit a 1T model into a phone. Makes much more sense to train a model that's designed to be small, than train a model that's smart and then quantize it into stupidity.

> * Chips with hardcoded transformer architecture

Totally, this will probably work great. Now good luck booking fab time any time in the next 2 years.

> * Much cheaper HBM

Totally, this will probably work great. Now good luck booking fab time any time in the next two years.

> * Much sparser models - 1T total with ~1-10B active params e.g.

Fewer active params helps with the speed of token generation, but if the whole model doesn't fit into ram it doesn't solve the issue of having to constantly stream portions of the model from disk to ram.

> * Not to mention - 2 years of today's frontier models writing RTL and kernels at superhuman levels.

IMO this is a delusional myth-making idea being sold to us by ai companies. Machines that generate output based on statistical averages won't generate genuinely new ideas. They can help us try out ideas faster, but they're simply not capable of the kind of creativity and understanding required to push a field forward, except incrementally.