Just writing this down so I can be praised/mocked in 5 years.
I'm fatigued by it all at this point. It's streamlining the interesting and fun parts out of my job (by practical necessity of use there), and if I used it half as much outside of work I'm sure it'd do the same there too.
This is the prevailing opinion of people even outside of tech.
That said, I think it's a good thing that this sentiment is coming to the forefront.
We are both late and early.
Buy an Nvidia Spark, then whatever cheap Mac you want to use as a thin client. There's no reason to force Apple Silicon's round peg into a square hole like AI inference.
You think this is a mistake...
Of course. Do you think this was on purpose? All part of Apple's brilliant master plan?
The one thing that is marginally exciting: the Apple SoC or M series chips.
It's unfortunate they are locked behind crappy macOS and other proprietary apple crap.
Unsurprising. Apple seriously thought the iPad would replace computers and usher in a "post-PC" word during their "What is a computer?" ad campaign era. Now they are sticking phone chips in laptop chassis.
It would need a path to a $2,500 machine, I think. But this is a niche I don’t think another consumer-facing brand could do like Apple.
Apple simply cannot comprehend the ask.
Apple knows the market demand for this type of device.
You may have paid $50,000 for it, but you’re only one customer. At Apple scale they need to focus their finite resources on the products that serve the largest market demand.
$50,000 rack mount servers are not a large demand.
It assumes that RAM remains supply constrained and that none of the existing RAM contracts are cut short.
But Meta and xAI putting A TON of AI compute onto the market. OpenAI and Anthropic are raising the costs of inference (by reducing how much inference users get via subscriptions). And we haven’t seen Oracle / CoreWeave struggle to pay their debts yet, but they will be selling assets once they get close to that point.
Chinese fabs might not be so tied with red tape and regulation upon regulation (which is a funny reversal, in terms of "communism vs capitalism" bureucracy/inefficiency cold war thinking)
All of their fabrication ability is based on old processes.
2.) Authoritarianism can move faster than anything. They can just say "wipe out that village, build the coal plant there, data center here, fab here.
3.) If it's red tape and regulation holding the US back, then that's clearly not "capitalism."
Except in the actual historical sense. They appear to enjoy all sorts of freedoms, increased prosperity, even have elections at different levels but under a single party system. Which is not necessarily that different than a effectively two party system.
>2.) Authoritarianism can move faster than anything. They can just say "wipe out that village, build the coal plant there, data center here, fab here.
Now that China is more effective, "it's easy because they're authoritarian". Before the argument was "authoritarianism can never be as effective as free-market democracy".
>3.) If it's red tape and regulation holding the US back, then that's clearly not "capitalism."
It's real world capitalism, not some fantasy some guy imagined removing all warts.
Unfortunately its not so cheap anymore as everyone ramped prices up of course.
Last year I could still get 32GB of DDR4 for under $60 from chinese brands.
I just upgraded my 2008 Thinkpad R61i to 8GB of DDR2 a few months ago while I was also upgrading to a core2duo.
DDR2 and DDR3 are still in active use by SBC manufacturers.
If a place can do it, another place, with a huge track record on manufacturing and lately expanding all kinds of tech, can.
Whether or not you feel like those are good overall (I do), they do actually also slow things down.
Yes, like how it helped western industry early on. Or, well into the 70s for the most part.
https://www.techspot.com/news/112502-memory-prices-tipped-fa...
what if demand keeps rising faster than production capacity is deployed?
We are in a bubble which will be burst the moment the world starts retaliating against the US' 20+ year history of supporting genocide and committing war crimes unabated.
Buy the AI toys while you still can.
Edit: Okay, this doesn’t mean that that’s actually possible in the short-term, so I think you’re right. But that means as the silver lining, in the medium term horizon there’ll be enough supply again? :’)
Memory is a cyclical market that has historically rewarded conservatism [1].
Counterpoint: there is enough demand from enough capital-rich customers that they may be willing to shoulder the capital risk.
[1] https://www.ldeepai.com/tech-hub/dram-industry-consolidation... Sorry for the slop link, it has a good chart from a solid source
This is a good hypothesis. Curious if anyone has data on the failure rates of new entrants in semiconductors based on how frothy it was on founding.
On one hand, more demand makes selling easier. On the other hand, a shortage makes your input costs (consumable and capital) pricier.
EDIT: It seems like the 2 to 3 year lead time and a crowding effect from new entrants historically made booting up a fab into a boom a bad bet [1]. (The article argues, convincingly, that this time may be different.)
[1] https://www.uncoveralpha.com/p/every-memory-cycle-ends-the-s...
So essentially, due to technological progress and other factors inducing price collapses (or at least cycles), you can’t start stockpiling insane amounts of finished-product semiconductor, which means you can’t scale production at current technology levels to infinity either?
Like, take out the price sheets for the Apple Car. Then sell me an AI tower at those price points.
The hard part is the GPU architecture. Apple Silicon was designed with a laser focus on raster efficiency (similar to AMD's GPUs) which makes a lot of sense for highly mobile hardware, but is a crippling mistake for high-performance compute. Apple's largest Ultra chips are hamstrung with SOC-tier GPU performance, their highest-end desktops are outperformed by Nvidia's laptop offerings. Apple has to find a way to scale upwards without imposing too much architectural strain on their cheaper hardware like the iPhone and Macbook. Nvidia has already solved this issue; full CUDA compute stacks are usable on extremely cheap GPUs like the Nintendo Switch's Tegra SOC, or the Mac Mini-sized Jetson boards.
In terms of "who needs to redesign more to address the market", Apple has a lot of technical debt to unearth before they catch up to Nvidia. And if they do catch up, Nvidia will still support Linux and other differentiating features that Apple refuses to implement. It definitely feels like Nvidia is closer to a winner with the Spark than Apple is with the Mini or Studio.
Those Macbook Neo users would be very reliant on Apple intelligence, enough maybe to pay for a service with it. I think Apple's much happier going this path.
If it's an "or," absolutely. But if it's an or, they should be prioritising Macbooks over the Mac Mini Doug Brooks is discussing.
When we breach the "and" of memory supply sufficient to allow for more Mac minis (and Mac Studios), I think it would make sense to consider relaunching Xserve (with new branding, of course) as a consumer/small business product.
The writing has been on the wall since 2019. Apple doesn't like the old way of computing, their goal is to expand the ecosystem by prioritizing install-base and then pushing first-party service offerings like they did with the iPhone. And like they did with the iPhone, Apple is great at ignoring power users to focus on features that make them more money.
You may be waiting a few decades for this type of product, memory supply be damned.
At the $150 mark (which is probably accurate factoring in lifetime service spend), that's a $10,000 minimum return on the 64x Macbook Neos. Apple can charge that type of premium on consumer hardware, but they're in no position to command $10,000 margins on professional hardware. They're not Nvidia, Apple has always been LARPing as an HPC vendor.
Now that the Mac Pro is depreciated, Apple's plan to pivot to service offerings seems set in stone. That's the "want it all" attitude they've adopted with the App Store.
I am unsure that apple themselves understand why their hardware (top end & bottom end) has been so successful, without this understanding leaning into these use cases isn't really going to be possible.
I trust they know more about their business model than some rando on the internet, sorry.
You have a bold career as a technical journalist ahead of you!
With their apple finger right there on the pulse, they are going hard on the VR/AR glasses (following the lead of the visionary CEO of facebook), cars and folding phones. By the end of the year (tm) we 100% will have all the features that were showcased and demonstrated 2 releases ago.
It's just that Apple isn't really focused on software development professionals, and it's still fashionable to throw shade on them, so we hear a lot of kvetching about it, in communities like this.
I've been using Macs for all kinds of stuff, since 1986, so I can definitely state they get work done.
But I still strongly believe that Apple hates pro users because they don't make as much money and because they get in the way of serving laymen. The Aperture fiasco, the Final Cut Saga, the Xcode war of attrition and the never ending chain of failures with MacPro - all suggest that I'm right.
(Developer of a major plugin for DaVinci here)
As an Apple developer, since then, I have been incandescent with rage at Apple, many times.
Guess I’m a walking demonstration of Stockholm Syndrome (at least, that’s what I’ve been told).
For me, the privacy pitch wins. I have a friend visiting, however, who spends like $2,400 with Anthropic every year. That's a solid ROI even if the thing becomes obsolete after a couple years. (I'm still on my 2020 MacBook Pro. I love it and will be sad when I have to replace it.)
How can that be a solid return on investment? There's no model you can run locally to have frontier model level performance. Also who spends 2.4k yearly for personal AI usage, like what's the usecase? If your friend is spending that money for his business then it's not personal computing.
I do the $100/mo for myself, then about ~$200/mo for startup.
I'm betting he doesn't need a frontier model. Sonnet, today, is likely good for 80% of his tasks, which largely involve repretitive, tedious work.
> Also who spends 2.4k yearly for personal AI usage, like what's the usecase? If your friend is spending that money for his business then it's not personal computing
Combination of business and personal.
I think "buy this $10,000 box and to easily grant every Macbook Neo on your team safe, private, free AI" could be a real winner.
Apple makes product lines with assembly lines, its not a hand fab or custom build type of place.
It seems like it's driven either by 1) people hearing Macs are good for AI, buying one, and using Claude for inference, not realizing that you interact with the anthropic API from an internet connected hair dryer. Or 2) people want their agents to have blue bubbles.
I find it hard to believe that enough normal people are doing on device inference is driving Mac Mini's out of stock. And even if they were the Mac mini is not actually a very good platform for it.
Neo-Siri in iOS 27 removes the need for a lot of this, but before then, if you want to ask a robot about information that is stored in Apple notes, or to send an iMessage, a Mac mini is your only practical option.
It has nothing to do with Macs being especially good at AI. It has everything to do with being one of the last 'cheap' devices being sold with that much unified RAM.
The second is that the puck is heading towards local models. The people running their own 'Claws are usually experimenting running their own services either to save money or to explore the future where 95% of requests are handled on device.
You can sort of justify it by assuming it will last a long time and they’ll use it for other things, too.
OpenClaw supports all the mainstream (and free) chat apps like Discord, WhatsApp, Signal, Telegram... None of them requiring a MacOS machine.
Is it a lack of knowledge from the users or do they really value iMessage integration that much?
The relevant questions here are: will the person using this machine also conceivably be wearing a pair of $549 AirPod Max? Or a $399 base Apple Watch? Does that person expect to pay more or less for their largest-screen computing device than their headphones?
Framing that way points toward a $350 price point being a laptop for young children (younger than Apple Watch age, so lower elementary). That's a whole different software experience beyond just the hardware.
Anyone who wanted the OpenClaw use case that is comfortable with Linux probably already has several Linux machines (including a few Raspberry Pis) on-hand.
My understanding is that the barrier to entry to using iMessage makes iMessage a LOT more secure from spam. If you want to do mass iMessages you have to register as a business with Apple, go through all sorts of checks and attestations, etc.
At any rate, iMessages are a lot more trustworthy than SMS. So being able to spam people via iMessage is very desirable. I recall a few months ago a guy posting his little spam-iMessage-as-a-Service product here on HN. You could build your little iMessage spam army using a bunch of Mac Minis...
I use Claude Pro ($20/m) as a glorified search engine (no ads/SEO) plus simple hobbyist dev things (shell scripts, managing my Mac, apps etc.
I also use it for tasks like - “search the web for top ten selling EVs, put them in a table” and then iterate - pivot tables, charts, additional research”. It could be cars, it could be broccoli. Code Work has facilities to streamline this type of work, but I usually drop into the CLI.
How much if any functionality would I need to recreate if I switch to OpenRouter and would be match my costs with the API approach. I don’t want any cost overruns. With Codex or Claude, if I run of tokens, no big deal, I can wait.
Thanks!
Unless you go for the very expensive options, most of the Mac Minis really aren't suitable for running local LLMs, they're painfully slow with prefill/processing input, and the models you are able to run don't handle long context very well, which these sort of long-running agents perform very differently with when you can.
I'll agree with your latter point, hard to beat the value of using something like OpenRouter or similar remote inference.
Even with local models, you can run the agent software and the inference workload on different hosts, which is what I'm doing at home. Beefy server responsible for inference, tiny VM on other server is running the actual agent software + RPC + bridges and what not.
No peripherals except Ethernet, integrated compute (cpu+gpu+mem) and secondary storage (+mobo, psu). No accoutrements, just the minimum amount of hardware to run a model as a utility.
Even the appliance faceplate would be a display showing stats like an old HiFi stereo.
Edit: something like a series of modules consisting of a RISC-V CPU + Vortex GPGPU + memory
Definitely on the edge of what would make sense at home, but its interesting.
> Unified memory in Linux creates a single address space accessible to both the CPU and GPU, eliminating the need to manually copy data between system RAM and video memory. It is enabled via NVIDIA's CUDA, AMD's ROCm/HIP, or generic kernel-level Heterogeneous Memory Management (HMM).
So it does exist and is available for platforms that matter.
Intel and AMD had been doing this for years already, and had linux support for it from day 1.
Otherwise, AMD is quite close to what Apple has, and Strix Halo is honestly incredible.
Not sure what RDMA brings to the table.
All have unified memory. Linux runs just fine on all of those.
Unfortunately their chatbot, while amazingly fast, doesn't know anything about the company running it.
Anyway I wouldn't mind an ASIC running a diffusion language model locally. Even if eventually it would become dated. Beats outsourcing all that to a company that's running on VC money which in the future might either perish or worse - dominate the market and charge whatever they wish.
That's a new one.
A bit too expensive for a home appliance though, isn't it?
95% of the price is going to be in GPU+CPU+RAM
Apps like LMStudio, Ollama, Draw Things, etc do a great job of simplifying it but it's still a pain.
This is mostly an US phenomenon, no Mac mini nor Mac Studio around here.
Only Thinkpads and Macbooks laptops talking to hyperscalers.
People are buying apple unified as electricity costs in many countries are very high, so cheaper to run than Nvidia setup.
As non-apple unified memory options increase, many people will have more choose those
> Many AI tools are also Mac-first or Mac-only
I fail to recall AI tools Mac-only general purpose AI or agentic tools. Most of the claws, harnesses, studios and inference engines seem to be multiplatform. You can say you can run then in a Mac with a nicer UI wrapper or whatever, but "Mac-first" or "Mac-only"?
[1] https://omlx.ai/
For as much as I dont like this aspect of modern computing, I understand why it is done from a technical perspective. Power, heat, and performance are all "better" when ram is on the motherboard vs in a "stick".
It’s not a huge niche but it’s an influential one. They’d get the engineers and CXOs of AI ventures and a lot of academics and hobbyists.
For the platform it would keep them cemented as the high end vendor. In the long term it would position them to take advantage of any software or training breakthroughs that deliver frontier model performance at that scale.
They would even sell less than Windows Server licenses.
By the way, they are down the same path with the workstation market, now that they only top level answer is the Mac Studio.
Workstation market wants flexible towers that they can customise to their own liking and special use cases.
The main reason Swift exists for Linux, is that app developers need to have servers somewhere, and if they want to share Swift code with the backend, well it isn't going to be on macOS Server.
but Apple needs to change the licensing model, currently you are allowed to run only 2 macOS VMs for every physical one you buy
So the ad free Apple on device experience will be welcome.
For example Apple radio is a free product with no ads, Apple TV and Apple Music don’t have an ads supported tier.
I said as they don’t push ads as much as others, the customer may be better
As ads focused services are designed to keep you there as long as possible rather than delivering what you are actually want quickly
1-2% is rounding error.
I don't think I'm taking this out of context when I say this is unintentionally correct. Apple still doesn't know what to do about AI.
Luckily, it doesn't matter because it's a solution in search of a problem. Most consumers aren't using AI apart from google search.
Everyone else is using it as a content scraper and praying nobody will step in to end the piracy/fraud.
This is... a view.
Maybe I live in a strange sphere of strange ("normie"-ish) people, but the people around me are for sure using AI. Mostly chatgpt to be fair. They use it to compare products that they intend to buy, identify plants in nature, create travel plans, find interesting places to visit nearby, give movie suggestions based on what they have previously enjoyed and so on and so forth. AI is becoming a very integrated part of their reality. To "google" something and digging through the search results manually is very rapidly being replaced by asking chatgpt, for better or worse.
I can see this kind of low level usage as being perfect for local LLMs... So I can't see a market there for openai etc forever.
Are they using the free chatgpt or a paid one?
SOTA AI for "serious" work is in a different position, used by fewer people but with big pockets and sometimes a pathological dependence on it.
>"I can't imagine where we're going to be a year from now, three months from now, or even a month from now,"
I'd say he's making an accurate appraisal of his abilities
Others running the beta now on newer iPhones and enjoying it more so?
On my Pro 16 it has its ups and downs - I still can't get it to "play my running playlist on shuffle" whilst running (this is the only thing I used Siri for before the beta and it would improve my life immeasurably if it worked). But it responds to things like "how long will it take to drive to the AirBnb booking in my inbox", and "when is X playing a concert in Y - add a calendar entry with details" perfectly.
This is a beta and I have hopes, but I can imagine it will run better on a 17 and later
Although given how effed up the voice for chatgpt is now with the latest updates I might talk with siri more.
Because I use carplay in tandem with my phone where the map is on the carplay screen and turn by turn directions are on my phone, it's always unlocked so I haven't run into whatever lock screen issue you brought up.
And the voice is still a poor text to speech model, very far behind GPT live.
That said, Siri seems a little bit better now - my subjective opinion. It is a little bit less frustrating.
> “He also described a shift toward running AI locally rather than in the cloud – a move motivated by privacy, security, and the rising cost of inference as agents consume more tokens.”
Classic Apple. No more just beating the “security and privacy” drum, now its “tokens are expensive!”
<neanderthal voice/> Cloud scary. Cloud expensive. Mac good. Buy Mac!
> “He also singled out what he calls ‘transparent AI’ on iPhone and iPad, referring to features scattered throughout the operating system and third-party apps that work quietly without announcing themselves as AI.”
<neanderthal voice/> Apple use AI, Apple just not say it. Apple smart, not lagging behind industry! Buy iPhone!
How about you invest in developing your own models, correctly? And provide a secure and private inference cloud service on your fancy Apple silicon? And integrate that into your platform so Siri gets smarter without you farming queries out to Google Gemini? Bill me for it in iCloud+ I’ll probably pay for those tokens.
Was that so hard?
AI features not being constantly shoved in my face and just selectively silently integrated where it’s most useful is preferred to what the rest of the industry has been doing, too. I think most of us are pretty sick of AI getting tacked onto things that don’t need it and then given prominent promotion and UI positioning, potentially at the cost of features we actually use.
They could be doing more, sure, but directionally this all seems fine?
Or phrase it in a very similar ask, why don't they invest in power plants? The model space is truly crowded, what do they gain or recover suppose they are SOTA? Across the Pacific they are pumping out free models that are only 6-12 months behind. What business sense does it make for Apple to develop their own models?
I agree, apple shouldn't invest in their own models. But they should have close to the best inference + end user design.
https://machinelearning.apple.com/research/introducing-third...
they just suck
These execs are so out of touch they believe Apple hardware to be "a system that's under their control", how does it come to this? Besides, a VM without bi-directional sharing of data gives you pretty much the exact same thing.
Did hundreds/thousands of developers really go out there and bought Mac Minis just because one prominent technology semi-celebrity happens to have used a Mac Mini for the development of their thing? Seems bananas people would spend hundreds on monies on something they barely grasp how it works.
And all of that because Tim Apple fears any feature that could mean people could have less than one iDevice per person.
Host resolution automatically matches that of client, image quality is great, framerate is decent, latency is minimal. The host creates virtual screens for the connection so connected screens don’t light up and the machine remains locked to anybody accessing it physically too, which is a nice privacy assurance.