I have an old M1 Max with 32GB that runs inference models fine, especially those using MLX.
Dedicated graphics cards are faster but not to the point that it matters.
That’s a six year old machine.
My newer machine M5 max 128GB will far outperform your typical 32GB gaming card once the model exceeds memory.
To even get close to that on a dedicated graphics card you are paying upwards of $20K.
That said, models are getting smaller and faster which allows me to run multiple different models with ease.
[edit] Just checked the 5 models I use take up 57GB when all loaded at the same time.
To be specific, 5 models at 57 gb means you are using crap quantized models, which suck for any real agentic work. I mean, sure they give you some inference, but compared to the full parameter models like Qwen3.8 and Gemma4 that can run full agentic loops, you may as well just use cloud inference for the price.
You of course could "run" those larger models, but we both know that the tok/sec is dogshit on Macs for those.
And 57 gb is split across 3 cards quite easily, which will all be cheaper than your comparable Mac and way faster.
You really need to educated yourself on how running local models works and what the models like Gemma 4 are capable of, so you don't continue to waste money on Macs.
It definitely matters, if you're intending to run a Claude Code/OpenCode style agent workflow. Most of those harnesses start with 8-12k token contexts, which is a lot of prefill for a Mac but cheap for a CUDA GPU. The GPGPU compute on the fastest Macs is still trailing behind Nvidia's laptop GPUs; the highest-bandwidth Apple Silicon chip (now the M5 Ultra) has ~7x lower memory bandwidth than a single B100 card.
There's a good reason why Apple Silicon isn't to be found anywhere in the datacenter buildout. It's nonviable for training, and wastes electricity running real-world inference workloads.