In order to get high bandwidth, you want memory as close as possible to the GPU. The more trace lane length = signal loss, bandwidth loss.. HBM is compact, and so you can stack 24GB modules, 8 around a GPU die.
If you tried to do that with normal memory, you need like 64 modules. So a a TON of traces more that all need to be equal length, and because so many = far away from the GPU = less bandwidth.
The issue is like stated above, its a process that waste a ton of wafers. Wafers that can make easily 3x more normal memory.
Intel with "Crescent Island" is trying to make a 160GB card using LPDDR5x memory but the bandwidth is only ~700GB/s.
The base interface die has eight or 12 memory dies stacked on top of it. The wafer cost of that interface die is therefore small.
If the process of die thinning and TSV stacking reduced yields by a factor of three, nobody would consider HBM mature enough to put into production, and especially not mature enough to be increasing stack height from one generation to the next.
Trace length has approximately nothing to do with die size. If anything, designing for shorter traces means you can get away with smaller PHYs at either end.
What might go some ways toward explaining such a huge difference in die size is that the TSVs themselves take up significant die area and must be fairly numerous to carry both a large number of signal wires and all the power and ground required by the stack. But it's wildly implausible that the die area consumed by the TSVs would be significantly larger than the die area consumed by the memory arrays themselves, or that anyone would build a memory die where the memory array was not a large majority of the total die area.
If there's any truth to that ~3x higher wafer requirement for the same number of bits as compared to DDR5, it must be a combination of several factors and probably includes something non-obvious and dubious, like counting all the area of the passive interposers that go between HBM stacks and GPUs (those interposers aren't competing for the same fab space).