upvote
It's not that there's a blocker. It's that it takes roughly 3x the manufacturing capacity to produce an HBM package at the same storage capacity as DRAM. We are sacrificing total bytes for bandwidth.
reply
I don't fully understand the source of the "total bytes" constraint, but a major factor may be because HBM4 / HBM4E can only make use of the footprint directly above the processor/logic die (or in direct vicinity of its interconnect), while traditional DRAM can be placed further away where there's lots of real estate on the motherboard.

I gather a practical max ceiling today is a stack of 16 chips in height yielding 64GB?

These chips have a massive bus size of 2048 bits, instead of the 64 or 128 bits (dual channel) used by DDR5. That's what gives them their order-of-magnitude bandwidth speedup. But even though they technically pack in more capacity per square millimeter of motherboard, I gather they take up more space than older technologies once you account for the vias and interconnects to route all those signals.

reply
Thanks for that, just went down an interesting rabbit hole. Many of us were hoping this re-tooling would eventually trickle some fast RAM down to DRAM-exhausted PCs, but given it would require a rearchitecture of the motherboard it's unlikely.
reply
How so? FEOL is pretty much the same, BEOL is almost the same save TSVs, the packaging tech is different and more advanced, but not exactly 1:1 comparable. Do TSVs really occupy 3x the area of DDR IO's?
reply
See remark on the slide 11: https://www.servethehome.com/micron-evolving-memory-architec... That presentation is by Micron.
reply
It doesn't really say that it needs to use 3x more area, but that 3x more gets consumed due to "advanced packaging and manfuacturing complexity". Which doesn't properly explain why it consumes 3x more and could simply mean they have a bad yield and 2/3 produced is garbage.
reply
> 2/3 produced is garbage.

This might not be far off the mark. You are irreversibly linking the fates of these devices after a certain stage of manufacturing. If something goes wrong at final packaging time, you lose all dies instead of one.

reply
Yea, that's the question. Yield situation can improve. Area overhead would not improve short of a completely new and incompatible tech.
reply
reply
Yuck. Time to build an xSPI/HyperRAM workstation (if only these had multi-bank chips).
reply
Sadly the $ per byte of xSPI and HyperRAM quite high
reply
Yes, but enough to run a text editor, or even a mechanical CAD. Not an LLM, but that's the point!
reply
3x is a reasonable figure. They are not literally that large, though.
reply
>> What's the main blocker (other than the current inflated cost) for using HBM instead of DRAM as the primary memory for consumer electronics?

HBM is meant to be integrated into the same package as the CPU, so no more DIMM sockets. It also has higher latency apparently.

reply
Mainly bus width. Afaik HBM is like 1024 bits vs DDRs 64 so you need lots of transfers in parallel to saturate the bus, and CPUs kinda want 64 bytes of data as that's the size of a cache line ASAP. So you need a ton of in flight transfers which isn't a thing CPUs provide, maybe multicore workloads.

Buy the way you win with CPUs is with latency, and not bandwidth, which is why Apple M series actually uses DDR with lower latency because of the stacking.

reply
I think people might prefer the lower idle power consumption from lpddr over the better bandwidth in hbm in battery powered stuff. That said right now the price is definitely preventing us from finding out.
reply
Idle yes, but HBM energy consumption / memory operations seems to be a bit better than DRAM.
reply
Only if you’re running at 100%. Consumers generally do not.
reply
That hasn't been true since early HBM2 days, before the controllers standardized on power/voltage management and did things like leave them in P0 to ship on time
reply
LPDDR/DDR/GDDR generally still win over HBM when there’s no data being transferred. HBM is primarily better in watts/byte transferred. Consumer electronics spend most of their time idle.
reply
Racing to idle is a very common power optimization technique
reply
Right, but HBM idle is significantly worse than LPDDR idle or even DDR idle for that matter. That matters a lot precisely because the device is idle most of the time. Your idle power draw dominates.
reply
Why would you want/need to?

The advantage of HBM over regular non-stacked DRAM is memory bandwidth, which also requires a super-wide memory bus - 2048 bits wide for HBM4. Compare that to the 128 bit wide bus of a modern CPU.

So to take advantage of it on the desktop, or anywhere else, you need that 2048 bit wide bus, and a processor capable of consuming 2-3 TB of data per second!

These are not normal requirements, other than for a GPU.

reply
SIMD (and especially the modern matrix extensions) can use as much bandwidth you can throw at it.
reply
Without further clarification, that statement seems impossible. "As much" being unbounded and all. You should expand what you mean.
reply
This will blow your mind, but it actually is pretty close to being unbounded. :)

Consider the 'MMA N matrices' primitive modern CPUs are starting to support. For the current generation of CPUs, N is a constant like 16 or 32, but there's nothing preventing it from being 1024 or larger if we have more memory bandwidth.

All this with a single instruction.

reply
Right but if HBM memory is all that people want to produce, then building a CPU which can use it use it would be useful on it's own merits.

But in reality we also already have unified memory architecture systems, integrated graphics etc.

reply
People want to produce hbm because it’s more expensive and more profitable than regular memory.
reply
There is a world where scale and experience means it's about the same though, is the thing.

And memory is already expensive. It's downright hard to even get it though - you frequently would prefer not what's cheapest, but whatever is in largest scale production.

reply
In one sense, nothing, in another, everything. It is DRAM, but the bandwidth requirements mean it’s paired to a processor, i.e. no DIMMs. Not 100% sure but things like MacBooks and the Framework tower, where you have fixed RAM for the device lifetime, have ~0 tradeoff.
reply
HBM is a stack of DRAMs, so there is no “instead”.
reply
The vias to enable stacking is a significant amount of the die area.
reply
If they're talking about production capacity, that is some product of die area and process steps, right? It doesn't have to be 3x die area, just 3x lower factory throughput for the same number of functioning memory bits.
reply
Nothing except CPU manufacturer choices. Mac laptops use it and they're consumer products.

People will have to get used to buying a fixed amount of RAM with their CPU but thats unlikely to be a problem.

reply
Apple's "unified memory" marketing is so strong that even tech literate people seem to have this misconception!

They have managed to pull this sort of thing off many many times. https://en.wikipedia.org/wiki/Reality_distortion_field

reply
Yup, the only reason Macs have higher memory bandwidth is because they use more memory channels, which gives them a wider bus. Both Intel and AMD only allow more than dual channel memory on server class processors these days.
reply
The “Apple only does X better because they do Y” thing has been a meme for ages. I remember dismissals like “PowerPC is only faster at math because it has more integer units” or something along those lines, and thinking, uh, isn’t that a good thing?
reply
There's a qualitative difference between "they're doing a different thing" and "they're doing the same thing, tuned differently". GP is saying that this is a case of "they just tuned it differently".

This distinction doesn't change what the performance numbers look like today, but it does inform what changes would be necessary for those numbers to look different tomorrow. E.g. Apple Silicon isn't fundamentally orders of magnitude more efficient than x86, they just used smaller features. Newer Intel and AMD chips made on equivalent processes _also_ get similar efficiency gains.

reply
Depends on the expense trade off.

If I get 10% more performance for 50% more cost it really depends on one's needs, for example.

reply
I am just saying it's not magic and x86 is capable of doing the same. Quad channel memory used to be more common in consumer hardware but now it looks like you don't even have the option anymore for desktops. AMD's Strix Halo was the first sign to reversing that (it has quad channel memory!) and hopefully we see more of that in the future.
reply
AMD does market segmentation and limits consumer chips to dual-channel. You need to cough up the dough and get Threadripper for quad-channel. Or even more for Threadripper Pro to get octa-channel.
reply
Afaik steam deck is quad channel, despite it being a pretty low end chip (with a decent GPU though)
reply
As I mentioned above AMD's Strix Halo has quad channel memory and is consumer level. But yes, other than that everything is segmented away.
reply
The memory bandwidth is a small thing compared to the massive win you get by not having to move data between two memory pools at all.
reply
Often you don't move memory around as a programmer, but that's exactly what happens in the background.

It's the address space that's unified, not always the physical hardware.

The data movement (when needed) is handled transparently in the background by page faults and other tricks.

reply
Depends on your workload. And AMD has supported unified memory long before Apple Silicon existed anyway.
reply
Except strix halo.
reply
Having unified memory is a real advantage though, it's not a reality distortion.
reply
The price is that you essentially glue CPU and GPU together, which limits total compute, from a size and thermal perspective.

This is really not a limit because of unified memory -- in principle, PCIe GPUs could read/write main memory without the CPU. But it's a limit for /fast/ unified memory, because fast means close.

So unified memory is great as long as the integrated GPU is strong enough. Then it has two advantages: a) probably faster transfer CPU<->GPU (but that's an implementation choice for the non-unified case b) If you either need a lot of memory for the CPU or the GPU, but not for both at the same time, you pay for memory only once.

reply
Agreed. That's why it's a good thing all computers made in the past 15 years have unified memory, not just macs.

https://x.com/Lina_Hoshino/status/1820947147312820497

reply
In theory perhaps but the benefit is not as relevant with a weak iGPU. In practice all PC's with performance ambitions had a dGPU until Strix Halo and Panther Lake.
reply
Is that Marcan's vtuber persona (Asahi Lina) that changed name since Marcan doesn't work on Asahi Linux anymore?
reply
It is a real advantage.

The reality distortion is that people seem to believe it's HBM, or somehow it gives you extraordinary amounts of vram. Neither are really true.

reply
MacBooks use regular soldered lpddr5(x) RAM. Same RAM as every other laptop manufacturer, they just use more lanes to achieve a higher bandwidth.
reply
Perhaps more noteworthy for general home and business computing, doesn’t it also allow for lower latency?
reply
More channels or soldered memory? Channels are basically RAID 0 so it depends what you're measuring. Soldering memory down was the only way to use LPDDR5X so if you wanted the best memory you had to solder it down. LPCAMM2 exists though so newer devices can use that instead of soldering them down, but not all devices would be able to fit the required LPCAMM2 slots.
reply
SOCAMM2 allows for removable ram in nearly 0 added space.
reply
> Mac laptops use it and they're consumer products

No, Mac laptops use LPDDR, currently LPDDR5X.

reply
The normal non-tech-savvy person already does this. They simply don't know that their ram is upgradeable or something else breaks first, before having to touch ram.
reply
what is this too much RAM thing you mention? I thought the only valid RAM situation you could find yourself is not enough RAM. Too much? That's just fantasy
reply
pretty sure its lpddr not hbm.
reply