Nothing stops you from giving Nvidia half a million for their latest and greatest AI GPUs, unless you are in China I guess, but China does very well despite the export control.
It is just classic market segmentation, the thing Nvidia and just about every company that can benefit from it has done for decades. Cloud providers are ready to pay a lot of money, consumers wanting local AI have a lower budget, and gamers even lower. But a low budget is greater than zero, and there is still money to be made on these markets. But they also don't want to compete against themselves, so they are going to find ways of crippling enterprise-class workloads on consumer hardware.
It used to be that they crippled high precision arithmetic on consumer hardware, because it was a differentiating factor, but AI doesn't care about high precision, so now, it is something else.
Pay up for license to unlock full capacity, pay up for license to use encrypted models from AI moguls, paid per billion tokens processed or just operations in GPU
I wonder if this space is explored fully enough at the moment.
Aside from some implementation details, it's conceptually much easier than doing it for a SQL database.
The consumer has access to free and open models for local use.
The supplier has better control and wider market to serve running the models in data centers.
Only way I can see encrypted models being widespread is if they are bundled as part of a software package (OS/Browser/etc) and not sold as the product.
AI is a software just like operating systems and executables where open source absolutely dominates the industry. Remember when the compiler engineers optimized the shit out of llama.cpp.
Right now the bottleneck for AI software is compute and data. Moore's law is give us compute, StackOverflow, arXiv (and it's children like bioRxiv), gutenberg and others (3lib, anna, internet.org) will give us data.
It's a waiting game. Open source always wins.
Tons of stuff happenning behind closed doors today though. No more stackoverflow for the next decade, that could create a problem if all new context only exists in chatgpt conversations only openAI has access to (or whoever).
So it is something that has been done for a long time now. Still feels like unethical rent-seeking that should not be tolerated by the customer.
See Also: car companies that charge additional fees for hardware already on the car. Additional services, I sort of understand(remote start via cell network), but stuff like heated seats, that is just wrong.
Yep, "Batterygate"
https://www.npr.org/2020/11/18/936268845/apple-agrees-to-pay...
It saves the car manufacturer on logistics cost. The per-unit cost of a seat heater is (much) smaller than the cost related to keeping two variants of seat in stock and to provide spare parts. Say the heating mat costs 100€ per seat including materials and installation, and can be sold to the customer for 400€ permanent, that means as long as 1 in 4 customers buys the seat heating option, the books are balanced, and all money you save on logistics is pure profit.
Adding subscription on top of that is financially even better, if there is one thing companies LOVE thanks to stonk market incentives it is monthly recurring revenue.
[Hanlon's Razor] This is such a contrived performance drop scenario that I find it hard to believe it was done with the purpose you're inferring.
I'd interpret this as Nvidia not caring that much about Linux desktops/laptops once again (as opposed to Windows desktops/laptops), rather than anything to do with local vs cloud.
As in, make your own inference and training hardware?
I stopped using my laptop for dev and switched to talking to claude code on my phone and as you can imagine I am faster at producing code.
100% cloud with a single entry point device to manage dedicated servers + agents