Now I want a local LLM server :( $25k is just so much money.
A lot more with Haiku equivalents (Qwen3.8 27B) or gemini. It's not frontier, but not that far away.
So much this. Plus everything else you mentioned. The freedom and control is real.
I've heard that there are legal software platforms like Clio that offer AI products, not sure what just eavesdropping on some lawyers at the bar.
Companies pretty much give us the full overview of what they have from a technical perspective (and often much more than that), we sign NDAs with investors that have serious teeth in them. The data they share with us would not be disclosed under any other circumstance and even our reports are shared only with the clients.
It's a nice little business. I'm not going to risk offending my customers for convenience or to save some money.
(Of course this all depends on if their plans work out, but if not then the smaller local models will probably play a big role in the failure of their plan)
At API pricing (i.e. what all enterprise customers are paying), inference is profitable, even if they're paying usurious rates on cost of capital for capex. Where the frontier labs are getting screwed is that they're in a Red Queen's race with each other, spending tens of billions of dollars per year training the next frontier model, and not being able to recoup the investment on that model before it's obsolete.
That's why you see the CEOs calling for government intervention to "pace the frontier": it lets them reduce investment on training without risking that their competitors keep going and make them obsolete. Unfortunately for them, the Chinese labs exist, and would love nothing more than to take the frontier, so they're in a bit of a pickle.
People keep repeating this, but it is only remotely close to true if you evaluate inference in a vacuum without any of the other supporting costs to facilitate that capability.
Just a few years ago people were saying the same thing about WeWork: "leasing is profitable, the unit economics of individual locations is net positive", and yet...
It doesn’t have to be black or white…
For my toy projects, adventures, learning exercises, and throwaway MVPs I will prefer the subscription plans… they are cheap for what they offer today.
It's training and tuning new models that cost them money. And that's why they are trying to scare people into thinking AI will kill us all in the next 5 years; they want bans on the open-weight models because they know they won't be able to compete with them.
If that is true, either the American labs are not optimizing correctly or they make a ton of money with inference.
The anthropic sub is likely still subventioned with VC money. So this may not even be available in 2 years.
But yeah, with current sub prices/budget... It's hard to go with anyone else unless you're rich.
The anthropic sub is likely still subventioned with VC money. So this may not even be available in 2 years.
But yeah, with current sub prices/budget... It'd stick with it too
They are subsidizing all of us to the tune of $billions in losses per month. That can't go on for a decade. Probably not even for another 18 months.
I still use all the tokens Claude will give me on the max plan. But I also run deepseek v4 locally for overflow and for... contingencies.
But, there is no need to make that bet - when prices increase, I'd look into renting a previous-gen GPU on a cloud provider. If the online GPU providers raise prices, then I would look into purchasing for on-site.
Until that happens, I'm not making any bets, because if all the token providers go bust GPU rentals are going to fall in price, and if they don't go bust after raising prices, I can still get years of GPUs for self-hosting at the price of a self-owned rig.
Just think of how user-hostile other online services have gotten, and imagine how acutely painful the proprietary LLM providers could make theirs. Thankfully, open models provide some counterbalance.
Are you comparing to subscription prices or API prices? Because if it's the former it won't last forever.