upvote
It is. That's the mac mini. For local LLMs you would want the Mac Studio, which tops out at 1.2TB/s.
reply
For max memory bandwidth you need to buy the Ultra versions (M5 Ultra: 1,2TB/s, this gets comparable to real GPUs regarding the memory bandwidth).
reply
It's higher bandwidth than any dual-channel DDR5 desktop machine, but Apple never quote the memory latency, so hard to compare otherwise.
reply
Would that be a difficult comparison to do fairly sine one is SoC and the other isn’t?
reply
I meant that it's a difficult comparison between platforms without having all the details like memory latency, how the memory controller load increases latency (can increase by 4x on some platforms), cache latency, TLB entries, etc.

Extremely high bandwidth is great for copying data, but not as relevant for walking chains of pointers, where latency/cache/TLB entries are more important.

So just saying "high bandwidth == better" is true when other variables are the same, but they rarely are, especially in comparison to x86-64 offerings.

All of these are independent that it is a SoC with on-package DRAM.

reply
It's fast by computer standards and excellent for entry level chip. The Pro/Max/Ultra chips are always faster.

Compared to something like VRAM it's slow.

reply
It's actually quite fast, more bandwidth than NVIDIA RTX 4090 at 1.0 TB/s according to https://www.digitalocean.com/community/tutorials/gpu-memory-... Compute power will still be different, though.
reply
The thread you're replying to is asking about the 170GB/sec memory bandwidth of the base M6, not the faster spec of the Ultra chips.
reply
M6 and M5 Ultra both have comparable memory bandwidth per GPU core. I think they will perform well.
reply
You would want to get the M5 pro version with 307gb/s if you were interested in running local LLMs.
reply
Kinda. Strix Halo does 256GB/s of memory bandwidth, and is significantly slower than an M5 Max (614GB/s). Feels like intentional market segmentation?
reply
You can get the m5 pro in the Mac mini with 307GB/s at 64gb of memory it’s $2899
reply