I have an old M1 Max with 32GB that runs inference models fine, especially those using MLX.
Dedicated graphics cards are faster but not to the point that it matters.
That’s a six year old machine.
My newer machine M5 max 128GB will far outperform your typical 32GB gaming card once the model exceeds memory.
To even get close to that on a dedicated graphics card you are paying upwards of $20K.
That said, models are getting smaller and faster which allows me to run multiple different models with ease.
[edit] Just checked the 5 models I use take up 57GB when all loaded at the same time.
It definitely matters, if you're intending to run a Claude Code/OpenCode style agent workflow. Most of those harnesses start with 8-12k token contexts, which is a lot of prefill for a Mac but cheap for a CUDA GPU. The GPGPU compute on the fastest Macs is still trailing behind Nvidia's laptop GPUs; the highest-bandwidth Apple Silicon chip (now the M5 Ultra) has ~7x lower memory bandwidth than a single B100 card.
There's a good reason why Apple Silicon isn't to be found anywhere in the datacenter buildout. It's nonviable for training, and wastes electricity running real-world inference workloads.