Yes, and they specifically mention "Up to 10.7x faster LLM prompt processing in LM Studio" which is probably using the neural accelerator for prefill.
replyHow much would is the cost for that machine though, I'm pretty sure I could just buy tokens from a provider and never run out of money for 10 years, and get far better quality output because inference is being served by professionals on far better hardware and this machine would be obsolete long before that as well. Hosting local seems like a possibly the dumbest thing you could possibly do from an economics perspective. And don't hit me with the privacy argument because everyone saying they care about privacy uses fucking gmail, whatsapp and instagram all day long.
replyYou can’t run multiple workers 24/7 for 100 bucks a month
replyOk sure. But this machine you can haul in an Ice Road Truck to the North Pole and do inference there in an off grid shack. Good luck talking to cloud AI there.
replyWow even the elves will get replaced by AI
replyYeah, that's who Apple is making these for fucking Santa and his Elves lmfao
reply[dead]
reply