It's notable that they're so valuable because they feature 800Gbps of memory bandwidth. About twice what's available on the top end of M5, and exactly what makes llm inference fast.
reply> because they feature 800Gbps of memory bandwidth. About twice what's available on the top end of M5
replyOuch, about half of the memory bandwidth of a dedicated GPU though :/ Running LLMs on Apple hardware still doesn't make any sense to me.
M5 max has 614GB/s, you mean the m4?
replyOh, I was looking at every M5 except for the 40-core M5 Max. They have 460.
reply