upvote
For local LLMs with a Mac, rule of thumb is you always want an Ultra (due to memory bandwidth). Even an M1 Ultra is superior to an M6 Pro in this regard.

There are no configurations even close to running something comparable to frontier model variants, they're simply far too large, but something like full precision Qwen 35b or DeepSeek 70b at 50+ t/s is well within available configuration, and potential for plenty of room for large context sizes.

reply
256GB is enough for DSv4 Flash, expect maybe ~30tg/s, and a lot better profile.

I'm using Flash heavily, and I would describe it as nearly as intelligent as Sonnet-class in agentic coding, but more usable. Less world knowledge of course, and definitely a bit less intelligent; but not _that_ much.

On usability: Takes less handholding, less likely to make unsolicited refactors or whatever, and the writing style is readable.

It's not great at super-long-horizon goals as the Claude 5 models are; but if you have a good harness, you can get around that.

reply
Opus is probably ~2T parameter model, so that would probably not run on these. More like Sonnet.
reply
The 512GB could run GLM 5.3 which is Opus level
reply
GLM 5.2 in NVFP4 is 465 GB. It would be a tough fit.
reply
Sonnet is estimated around 1T, so that is far beyond what's practical as well.
reply
With the newest Qwen 3.8 27B model you can get opus on just about any new Mac.
reply
It can fit into 16GB macbook ?
reply
deleted
reply
[dead]
reply