The maths don't check. With Deepseek Flash one goes a very long way with 1600$ - even 10$/month, for easy jobs, are more than 13 years, and at a higher quality.
The low hanging fruit stuff for me is more something I use it for because I have the local LLM setup running anyway. It wasn't the reason I bought it, but now that it's there I might just as well use it as much as I can.
I think that some of the hardware design folks have been blindsided by AI demand and we haven’t really gotten that next generation AI hardware yet, to the point where buying M5 isn’t going to make sense in a couple of years.
Rumors seem to be that the M7 is the generation that Apple is looking to push AI performance much further.
I’m not sure that Apple anticipated this specific route that computer hardware has gone and I don’t think M5 and previous iterations were really specifically architected for local AI performance, more like they happened to be pretty good at it.
Big upgrade to memory capacity but memory speed is only going up by a few percent, so its still going to be slow with more than a few B active params (I have one)
- There's no guarantee of the $20/month service, and it likely has some limits compared to dedicated hardware token wise.
- Model are becoming more and more efficient, in many cases an M1 Max Mac Studio is still capable with 32 GB. 128 GB ram may not be the necessary baseline.
- Folks may think they want to only have a general model running locally (it's the comparable after all from the cloud providers), but we have to remember if the tasks we're trying to do ultimately are more specific than general and if there's space for the smaller models to do that.