SIMD (and especially the modern matrix extensions) can use as much bandwidth you can throw at it.
replyWithout further clarification, that statement seems impossible. "As much" being unbounded and all. You should expand what you mean.
replyThis will blow your mind, but it actually is pretty close to being unbounded. :)
replyConsider the 'MMA N matrices' primitive modern CPUs are starting to support. For the current generation of CPUs, N is a constant like 16 or 32, but there's nothing preventing it from being 1024 or larger if we have more memory bandwidth.
All this with a single instruction.
Right but if HBM memory is all that people want to produce, then building a CPU which can use it use it would be useful on it's own merits.
replyBut in reality we also already have unified memory architecture systems, integrated graphics etc.
People want to produce hbm because it’s more expensive and more profitable than regular memory.
replyThere is a world where scale and experience means it's about the same though, is the thing.
replyAnd memory is already expensive. It's downright hard to even get it though - you frequently would prefer not what's cheapest, but whatever is in largest scale production.