upvote
But where are you going to find an Nvidia GPU with 128+ GB of memory at an enthusiast-compatible price?
reply
You don’t need it if you use llamacpp on Windows, or if you compile it on Linux with CUDA 13 and the correct kernel HMM support, and you’re only using MoE models (which, tbh, you should be doing anyways).
reply
What MoE has to do with it? Aside from Flash-MoE that supports exactly one model and only on macOs - you still need to load entire model into memory. You also don't know what experts going to be activated, so it's not like you can predict which needs to be loaded.
reply
That might even be true, but how large is the TAM for such machines?
reply
Some Chinese sources sell modded Nvidia GPUs with extra VRAM. They're quite affordable in comparison to even a Mac Pro.
reply
Any links to them? Never heard of this..
reply
I've seen a guy who sells modded 2080 Ti with 22gb for $500

https://www.tomshardware.com/pc-components/gpus/chinese-work...

There's also unreleased Nvidia engineering samples of cards with doubled VRAM like this - https://www.reddit.com/r/nvidia/comments/1rczghu/update_unre...

reply
It’s been going on for a while. Search YouTube or the web for 48gb 4090 (this is one of the most popular modded Nvidia cards), Nvidia of course never officially made a 4090 with this much memory.

There are some on sale via eBay right now. The memory controllers on some Nvidia gpus support well beyond the 16-24gb they shipped with as standard, and enterprising folks in China desolder the original memory chips and fit higher capacity ones.

reply
Go at ebay and search for RTX 4090 48GBs. There's plenty of them with prices around $3.5k
reply
And how much do you trust Chinese hardware?
reply
Give that most of mine, and probably yours, and probably most of the world's computers are in fact made in China one way or another, some higher percentage than others, I'm guessing most of us trust our hardware enough to continue using it.
reply
When there's no one left to trust, maybe you need to re-evaluate your criteria.
reply
I wouldn't say that's true or even likely. It's completely possible to be in a pit of vipers where every single snake is venomous, and that is pretty much what we are seeing: With technological advances, there is a certain subset of people that will use them primarily to solidify their power and control over others. There is no utopian society right now whose government doesn't look to spy through technology, which of course is best set up at time of manufacture.
reply
Agreed. Unless you have full control over the production chain to fully produce a device, you are subject to the whims and desires of those who preside over such technological feats that we take for granted in our daily lives.

To the original point, it's safe to say that highlighting a nationality with regards to trust is baseless and without merit, as would be for any other topic (men/women from x are y, z food is better here, etc..). Real life is much more complicated and nuanced past nationalities. Some might call it FUD (fear, uncertainty and doubt) but there's always a deeper rationale at the individual level as well.

reply
Rather than people being wary of Chinese in general, it's more that there is a high degree of government control exercised in China and they are known to be very strategic with long-term planning in regards to technology control both for spying and actual remote control of devices. We are all just looking for the least bad option. It's not like devices from other countries are immune, but they are often less organized so there is a better chance of avoiding the Chinese level of planned access.

It does seem like pretty low risk in this specific case so I agree OP's comment was bit over the top, but I would have no way to make anything resembling even an educated guess as to how far their programs go.

reply
The Mac is also chinese hardware
reply
It would be hilarious if you are using a Lenovo device right now.
reply
and let alone competing on the energy consumption!
reply
The Nvidia DGX Spark is exactly this and in the same price and performance bracket.
reply
Sadly, memory bandwidth is abysmal compared to Apple chips - 273 GB/s vs 614 GB/s on M5 Max for similar price. Even though fp4 compute is faster, it doesn't help for all the decode heavy agentic workflows.
reply
You can still buy used 3090 cards on ebay. 5 of them will give you 120GB of memory and will blow away any mac in terms of performance on LLM workloads. They have gone up in price lately and are now about $1100 each, but at one point they were $700-800 each.
reply
I don't see how 5x 3090's is a better option than an M3 Ultra Mac studio.

The mac will just work for models as large as 100B, can go higher with quantized models. And power draw will be 1/5th as much as the 3090 setup.

You can certainly daisy chain several 3090's together but it doesn't work seamlessly.

reply
> You can certainly daisy chain several 3090's together

It's not "daisy chaining" 3090 has NVLink.

reply
Really? How would you NVLink more than 2 3090's?
reply
> The mac will just work for models as large as 100B, can go higher with quantized models. And power draw will be 1/5th as much as the 3090 setup.

This setup will work for 100B models as well. And yes, the Mac will draw less power, but the Nvidia machine will be many times faster. So depending on your specific Mac and your specific Nvidia setup, the performance per watt will be in the same ballpark. And higher absolute performance is certainly a nice perk.

> You can certainly daisy chain several 3090's together but it doesn't work seamlessly.

Citation needed; there's no "daisy chaining" in the setup I describe, and low level libraries like pytorch as well as higher level tools like Ollama all seamlessly support multiple GPUs.

reply
I think it's bad form to say "citation needed" when your original claim didn't include citations.

Regardless - there's a difference between training and inference. And pytorch doesn't magically make 5 gpus behave like 1 gpu.

reply
How much does it cost to have an electrician wire up 240v circuit just to power the thing?
reply
The machine I’m describing works just fine on a dedicated 15A 120V circuit.
reply
Where are you gonna find Apple hardware with 128GB of memory at enthusiast-compatible price?

The cheapest Apple desktop with 128GB of memory shows up as costing $3499 for me, which isn't very "enthusiast-compatible", it's about 3x the minimum salary in my country!

reply
Apple is not catering to minimum salaries in poor countries. Does this really need to be explained?

$3499 is definitely enthusiast compatible. That's beefy gaming PC tier, which is possibly the canonical example of an enthusiast market.

This isn't tens of thousands of dollars for top tier Nvidia chips we're talking about.

reply
Seems I misunderstood what a "enthusiast" is, I thought it was about someone "excited about something" but seems the typical definition includes them having a lot of money too, my bad.
reply
I'm an immigrant to Canada, and yes, English has both literal meanings and colloquial meanings.

In the most literal meaning, absolutely, "Enthusiast" just means a person who likes something, is excited about something.

When it comes to market and products though, typically you'll see the word "Enthusiast" as mid-tier - something like: Consumer --> Enthusiast --> Professional (may have words like "Prosumer" in there as well etc:)

In that context, which is typically the one people will use when discussing product pricing and placement, "Enthusiast" is somebody who yes enjoys something, but does it sufficiently to be discerning and capable of purchasing mid-tier or above hardware.

So while a consumer photographer, may use their phone or compact or all-in-one camera, enthusiast photographer will probably spend $3000 - $5000 in camera gear. Equivalently, there are myriad gamers out there (on phones, consoles, Geforce Now, whatever:), an enthusiast gamer is assumed to have a dedicated gaming computer, probably a tower, with a dedicated video card, likely say a 5070ti or above, probably 32GB+ RAM, couple of SSDs which are not entry level, etc.

Again, this is not to say a person with limited budget is "not a real enthusiast", no gatekeeping is intended here; simply, if it may help, what the word means when it comes to market segmentation and product pricing :)

reply
Additionally, "enthusiasts"/"hobbyists" tend to be willing to spend beyond practical utility, while professionals are more interested in pragmatism, especially in photography from what I can tell.

If you're an actual pro, you need your stuff to work properly, efficiently, reliably, when it's called for. When you're a hobbyist, it's sometimes almost the goal to waste money and time on stuff that really doesn't matter beyond your interest in it; working on the thing is the point, not the value it generates. Pros should spend money on good tools and research and knowledge, but it usually needs to be an investment, sometimes crossing over with hobbyist opinions.

A friend of mine who's a computer hobbyist and retail IT tech, making far far less than I do, spends comically more than me on hardware to play basically one game. He keeps up to date with the latest processors and all that stuff, he knows hardware in terms of gaming. I meanwhile—despite having more money available—have a fairly budget gaming PC that I did build myself, but contains entirely old/used components, some of which he just needed to get rid of and gave me for free, and I upgrade my main mac every 5 years or something. I only upgrade when hardware is really getting in my way.

reply
>> So while a consumer photographer, may use their phone or compact or all-in-one camera, enthusiast photographer will probably spend $3000 - $5000 in camera gear.

It's interesting that you chose photographers as the example here. In many cases that I've seen, enthusiast photographers spend much more than professional photographers on their gear because the photographers make their money with their gear and therefore need to justify it, while the enthusiasts are often tech people, successful doctors, etc., who spend lots and lots on money on their hobbies...

In any case, your point stands, that "enthusiast" computer users would easily spend $3-4K or more on gear to play games, train models, etc.

reply
$3.5k is a lot of money, but not a ton by American hobby standards. It's easy to spend multiples, even orders of magnitude more than that on hobbies like fishing, wine, sports tickets, concerts, scuba, travel, being a foodie, golf, marathons, collectibles, etc.

It's out of reach for lots of people, even in developed countries. But it's easily within reach for loads of people that care more about computing than other stuff.

reply
In June 1977, the base Apple II model with 4 KB of RAM was $1,298 (equivalent to about $6,900 in 2025), and with the maximum 48 KB of RAM it was $2,638 (equivalent to about $14,000 in 2025).

(Source: Wikipedia via Claude Opus)

reply
Wow, 48k for $14000. Now you can get a MBP with a million times more memory for $3500 or so. Whereas that CPU was clocked at 1 MHz, so CPUs are only several thousand times faster, maybe something like 30,000 times faster if you can make use of multi-core.
reply
I'd argue that some of those are more consumption and activity than hobby depending on how they're engaged with, and that people use the word "hobby" too loosely, but would agree that Americans in-particular consume at obscene rates.

Golf equipment, mountaineering equipment, skiing and snowboarding lift tickets and gear, a single excessive graphics card that's only used for increasing frame rates marginally, or basically a single extra feature on a car, are all things that accumulate quite quickly. Some are clearly more superfluous than others and cater to whales, while some are just expensive by nature and aren't attempting to be anything else

reply
Those are the prices for just buying equipment, which at least retain some kind of value. 3 million+ American kids are enrolled in competitive soccer with annual clubs dues between $1K and $5K, and that money is just gone at the end of the year. Basically none of those kids are going to have a career in soccer, so it's clearly a hobby, and everyone knows it. And soccer isn't even the most popular sport!
reply
Ya, I guess that's another category entirely. The cost of enrolling a kid in anything, potential travel involved etc..
reply
I live in America, I am very well compensated. Have been for 15 years now. $3500 is a lot of money. A lot. There is a tiny bubble of us tech folks who think it is accessible to most people. It is not. It is also the same reason Macs are still a niche. Don't take your circles to be the standard, it is very very far from it, especially if you think $3500 is not a lot of money.

It is easy to confirm this, just look at the sales number of these $3500 devices. It is definitely not an enthusiast price point, even in the US.

reply
It's not nothing for most people... it's more than a month of rent/mortgage for a significant number of Americans even. But if it's your primary hobby, it's not completely out of reach, and it's not something you necessarily spend every year. A lot of people will upgrade to a new computer every 3-5 years and maybe upgrade something in between those complete system upgrades.

I know plenty of people who don't make a lot of money (say top 25% or so) that will have a Boat or RV that costs more than a $3500 computer, and balk at the thought of spending that much on a computer. It just depends on where your interests are.

reply
The first words I said: "$3.5k is a lot of money..."

There are tens of millions of top 10% income adults in America. So something can be both unaffordable to most people, and also easily accessible to very many people.

reply
It’s a midrange to upper expense in the US if it’s your hobby. Most people don’t have a serious computer hobby but they golf, trade ATVs, travel, drink, etc.
reply
There are something like 24 million millionaires in the United States... Estimates are that Americans spent $157 billion on pets in 2025.

There are a lot of people who could easily choose to spend $3,500 on a computer.

reply
$3500 would have been 3–4 months' discretionary spending as a PhD student in Finland 15 years ago. A sum you might choose to spend once a year on something you find genuinely interesting.

Some people succumb to lifestyle creep or choose it deliberately. Others choose to live below their means when their income grows. The latter have a lot more money to spend on extras, or to save if that's what they prefer.

reply
An enthusiast in the hobby space is by definition someone willing to pour much more money that someone else not that enthusiast in whichever hobby we are talking about.
reply
Well, and also has a bunch of money, not just willing. I guess locally we don't really have that difference, as two other commentators here went by, that's why I had to update my local understanding of "enthusiast". Usually we use it for how engaged/interested a person is, regardless of how much money they can or are willing to use.

Learned something new today at least, so that's cool :)

reply
Yes, when tech gear is sold as 'enthusiast' gear, it is almost invariably the most expensive non-professional tier of equipment. That is roughly the common understanding: Expensive and focused on features more than security required for public use; while remaining within reach of at least some individuals, not only corporations.
reply
In a hobby where there are (strong) HW requirements, it mostly takes for granted you have money to shell out for your hobby, indeed.
reply
For an individual making median income in the US, it would cost 2% of your income to get a machine like this every 4-5 years. That's a matter of enthusiasm, not a matter of having a lot of money. Sorry that income is less where you are, but the people talking about the product tier are using American standards.
reply
1200$ as the minimum salary covers probably 70% of Europe by population?
reply
The Neo has enough power to do small LLM testing and pretty much anything else a bit slowly, and costs $600?
reply
Neo tops at 8GB RAM. What LLM are you going to run there? Functiongemma?

It can absolutely do some ML inference on it, but not much in terms of LLMs.

reply
Did you need to add poor? Unless apple isn't catering to the US
reply
I spent aaround that on my current personal desktop... 9950X, 2x48gb ddr5/@6000, RX 9070XT, 4tb gen 5 nvme + 4tb gen 4 nvme. I could have cut the cpu to a 9800x3d and ram to 32gb with a different GPU if my needs/usage were different. I'm running in Linux and don't game too much.

That said, a higher end gaming setup is going to cost that much and is absolutely in the enthusiast realm. "enthusiast" doesn't mean compatible with "minimum wage"

reply
The original Mac with 128KB of memory cost $2,495 when Apple released it in 1984. It would be about 3x that in today's money.
reply
I came here to say the same. Even with my student discount price of $1000, that's over 3K in today's dollars.

We are so freaking spoiled by the cheap cost of compute now.

reply
> it's about 3x the minimum salary in my country!

Enthusiast compute hardware doesn't cater to the people on the minimum salary in any country, let alone developing nations. When Ferrari makes a car they don't ask themselves if people on minimum salary will be able to afford them.

In in the bottom two poorest EU member states and Apple and Microsoft Xbox don't even bother to have a direct to customer store presence here, you buy them from third party retailers.

Why? Probably because their metrics show people here are too poor to afford their products en-masse to be worth operating a dedicated sales entity. Even though plenty of people do own top of the line Macbooks here, it's just the wealthy enthusiast niche, but it's still a niche for the volumes they (wish to)operate at. Why do you think Apple launched the Mac Neo?

reply
Right, I think maybe we're then talking about "upper class enthusiasts" or something in reality then? I understood that to juts be about the person, not what economic class they were in, maybe I misunderstood.
reply
Yes, it's a different definition.

Enthusiast in this contest more or less means you are excited enough about something to get a level above what normal people should get and just below professional pricing. An enthusiast camera body can be 2000 euros.

I would say an enthusiast computer is 2-4k.

It really depends what you meant with minimum salary (yearly?) because paying 3 months of salary for a computer like that isn't far fetched. You're not using this to generate recipes for cookies. An enthusiast level car is expensive as well.

reply
deleted
reply
enthusiasts in computer hardware assumes enthusiasm about hardware, not about "hardware on an budget". It doesn't matter if it's afforable or not.
reply
>Right, I think maybe we're then talking about "upper class enthusiasts" or something in reality then?

Why? Enthusiasts are by definition people for whom value for money is not the main driver but top performance and cutting edge novelty at any cost. Affording enthusiast computer hardware is not a human right same how affording a Lamborghini or McMansion isn't.

But you don't need to buy a Lamborghini to do your grocery shopping or drive your kids to school, same how you don't need an Nvidia 5090 or MacBook Pro Max to do your taxes or do your school work.

So the definition is fine as it is. It's hardware for people with very deep pockets, often called whales.

reply
tell me what pc with an nvidia gpu can you buy with same memory and performance.

I never liked apple hardware, but they are now untouchable since their shift to own sillicon for home hardware.

reply
> tell me what pc with an nvidia gpu can you buy with same memory and performance.

And power consumption !

The performance per watt of Apple is unmatched.

reply
This needs to be sold as the big ticket item for low level devs. Their chips are some of the most power efficient chips on the market right now.

Hoping they release a blade server version somehow.

reply
Apple releasing anything enterprise or "server" related would be a pretty big pivot - let alone blades.
reply
Nvidia's recent GPUs are more power-efficient than Apple Silicon in raster, training and inference workloads.

A blade server would get cancelled just like the Mac Pro for exactly the same reasons: https://9to5mac.com/2026/03/02/some-apple-ai-servers-are-rep...

reply
> Nvidia's recent GPUs are more power-efficient than Apple Silicon in raster, training and inference workloads.

I think you can do better than the proverbial Apples and Oranges comparison.

In terms of total system, "box on desk", Apple is likely to remain the performance per watt leader compared to random PC workstations with whatever GPUs you put inside.

reply
Then ignore me, and go ask your local datacenter why Apple Silicon isn't on any of their racks.
reply
I've owned some beefy computers in the past and this tiny little m4 mini on my desk blows them all out of the water easily. It's crazy.
reply
Untouchable my ass. You get a PC that has an ssd glued to the motherboard so if you run write intensive workloads and that thing wears out replacing it will have significant cost. Then there’s no PCie slot to get any decent network card if you want to work more than one of them in unison, you’re stuck with that stupid thunderbolt 5 while Infiniband gives x10 network speeds. As for memory bandwidth, it’s fast compared to CPUs but any enterprise GPU dwarfs it significantly. The unified RAM is the only interesting angle.

Apple could have taken a chunk of the enterprise market now with that AI craze if they had made an upgradable and expandable server edition based on their silicon. But no, everything has to be bolt down and restricted.

reply
This has changed since Sam Altman started buying up all the chip supply, raising prices on memory, storage, and GPUs for everyone, but it used to be the case that you could build a PC that was both cheaper and faster than a Mac for LLM inference, with roughly equal performance per watt.

You would use multiple *90-series GPUs, throttled down in terms of power. Depending on the GPU, the sweet spot is between 225-350W, where for LLM workloads you only lose 5-10% of performance for a ~50% drop in power consumption.

Combined with a workstation (Xeon/Epyc) CPU with lots of PCIe, you can support 6-7 such GPUs (or more, depending on available power). This will blow away the fastest Mac studio, at a comparable performance per watt.

Again, a lot of this has changed, since GPUs and memory are so much more expensive now.

Macs are great for a simpler all in one box with high memory bandwidth and middling-to-decent GPU performance, but they are (or were) absolutely not "untouchable."

reply
With 6-7 GPUs and EPYC cpu it will also cost 2-3x more than a Mac Studio.
reply
I think OP’s point was that it would do more than 2-3x the workload, thus them stating “blow it out of the water” and specifying “performance-per-watt”.
reply
But they're pretty fast and can have loads of RAM, which would be prohibitively expensive with Nvidia.
reply
A 128GB 2TB Dell Pro Max with Nvidia GB10 is about $4200, a Mac Studio with 128GB RAM and 2TB storage is $4100. So pretty comparable. I think Dell's pricing has been rocked more by the RAM shortage too.
reply
Unfortunately the GB10 is incredibly bandwith starved. You get 128gb ram, but only 270GB/s bandwidth. The M3 Ultra mac studio gets you 820GB/s. (The M4 max is at 410GB/s. I'm not aware of any workload that gets the GB10 to it's theoretical peakflops.
reply
You can't get a 128GB M3 Ultra, it's also more expensive. For some workloads the Studio is better, for others the GB10.
reply
~not unified memory tho~
reply
It is unified memory on this one
reply
From the spec sheets I’m looking at, it is not. I’m seeing models of the Dell Pro Max with 128 GB of DDR5-6400 as CAMM2, then a separate memory of up to 24 GB on the GPU. CAMM2 does not make the memory unified.

There are also SO-DIMM options.

reply
You're not looking at the right thing. Dell's naming is horrible. Dell Pro Max with GB10 (https://www.dell.com/en-us/shop/cty/pdp/spd/dell-pro-max-fcm...). It's a very different computer than what you're looking at and has 128GB LPDDR5X unified memory.
reply
Thanks for pointing that out. I found a more informative article about that model at https://www.mcpgov.com/dell-pro-max-gb10
reply
I took ~ to be a "singing tone" for some reason till I saw sibling and realized it might be an attempted strikethrough xD
reply
That won't hold much benefit as SOCAMM2 and LPCAMM2 get more popular.
reply
> So pretty comparable.

The Mac Studio almost certainly uses at least half the power

(educated guess, I'm too lazy to go look at all the spec sheets and run the numbers)

reply
It's actually reversed. The GB10 chipset has a TDP of 140w, whereas M2/M3 Ultra pulls over 250w from the wall: https://support.apple.com/en-us/102027
reply
> It's actually reversed. The GB10 chipset has a TDP of 140w, whereas M2/M3 Ultra pulls over 250w from the wall

Come on mate ... I think you and I both know I was talking about complete system here, not discrete components.

I'm pretty sure your total package (Dell Pro Max + GB10) will pull more from the wall.

reply
I'm pretty sure you need to look up what you're talking about instead of making a guess.

The Dell Pro Max PSU + enclosure is only rated for 240w, it literally can't pull more than 250w from the wall without shorting itself.

reply
> 240w

280w according to the spec sheet I just looked at.

Also just look at the graphs on Geerling's website. The Mac Studio eats the Dell for breakfast in a number of the tests: https://www.jeffgeerling.com/blog/2025/dells-version-dgx-spa...

reply
Not quite, what is the vRAM bandwidth of each? The bandwidth is a huge contributor to LLM performance.
reply
AFAIK, for the unified bandwidth, it depends mostly on the CPU, for M4 Max (I think it's the default today?) it does ~550 GB/s, while GB10 does ~270 GB/s, so about a 2x difference between the two. For comparison, RTX Pro 6000 does 1.8 TB/s, pretty much the same as what a 5090 does, which is probably the fastest/best GPUs a prosumer reasonable could get.
reply
Do NVIDIA solutions also outperform the Apple M-series in performance per Watt?
reply
No, that's why Apple uses Performance Per Watt not actual performance celling as the metric. In actual workloads where you'd need this power then actual performance is what matters not PPW.
reply
Probably comparable, but that's only with business-grade products, it's why Apple's current silicon is so remarkable on the market at the consumer level.
reply
Thanks.
reply
Nvidia isn't selling one-off home computers afaik. But yes in terms of datacenter cloud usage Nvidia performs.
reply
GB300 DGX Station was announced last Monday.
reply
It's going to cost far more than a diy machine with multiple lower end GPUs. Which is fine -- it's aimed at enterprise, not home labs.
reply
reply
Amusingly there's a macbook next to it in the pic, is this headless?
reply
It has a HDMI port and its USB-C ports also support display out. But I believe most who buy it intend to use it headless. The machine runs Ubuntu 24.04 and has a slightly customised Gnome (green accents and an nvidia logo in GDM) as its desktop.
reply