Anyone with an M-series Apple computer can run something very competently. Mac Pro users can run 30B class models which is good enough for the vast majority of practical everyday purposes, far better than the original ChatGPT was. Anyone with a gaming computer is in a similar situation. The rest of us can still run stuff, just not as big or as fast.

They have it, we just haven’t enabled them. The smart model with a chat box is the wrong abstraction for local. Ideally we would have it built into applications as a clear and easy to use opt-in feature. Like allowing a user to index a folder on their hard drive and then search it semantically via embeddings. You could do that on fairly low end hardware these days. Like 2GB of RAM with any processor made within the last 10 years.

They may not right now, but the whole point of Microsoft's Copilot+ PC standard (even though it's somewhat anemic) is to run models locally. Apple Silicon with enough unified memory is capable. Not to mention modern iPhones and Pixels have fairly capable NPUs and routinely run local models. So, we may not be to the point where most normal people have the hardware to run local models, but it is rapidly approaching.

As time goes on, they’re almost certainly will be very capable local models in the long run we (general computer users) aren’t going back to the era of mainframe computing no matter how much OpenAI, Meta or Google would like us to.

Gamers can run Qwen 3.6 quantised models now.

You would also be shocked what's possible on a 64GB Mac Studio, which isn't that unattainable.