With the size of these matrices I don't think they are even meaningfully colocated with themselves in memory. You'll end up with some dataflow TPU architecture anyway because you'll have to stream the second matrix to multiply against.

Turning RAM into an active coprocessor does feel like an awkward model for that though.

Intel's Advanced Matrix Extensions (AMX) and the dedicated tile registers, which are basically systolic arrays IIUC, seem like a better fit.

But Samsung doesn't have that option so they're playing the card they have.

Exactly where I see this going as well. Sure, a smartphone might be a nice place to introduce such tech. But matrix multiplication is literally where all non-labour jobs are going - this is the bedrock for efficient (time, energy) machine learning and inference.

If AI really is going to eat all our jobs, then matrix multiplication in memory is almost a requirement.