I get (sometimes) frustrated with people that don't work in the tech industry over this concept, and having them rationalize that their usage is heavily subsidized by the distributor of the model.
The costs in compute, electricity, cooling, etc. per token is much higher than most people (I feel) who work in the industry realize. Which, cognitively makes sense: they just want the 'magic 8-ball' to give them a tasty dinner recipe that meets their specifications, for example, and don't care how the 'sausage' is made.
However, economies generally equalize with supply and demand, and this (most likely) will become cheaper as the technology progresses and optimization passes/new algorithms are discovered.
It seems likely that Anthropic and OpenAI's inference margins on API prices are >95%.
That also casts doubt on the parent's claim that subscriptions are subsidized. It rather seems almost certain that they are overall generating profit.
My last role swapping from Haiku/Sonnet to GLM dropped costs by 90-95% and increased quality.
Money talks.
I can understand that they might be cautious about sharing their data with a Chinese company.
But them being the most successful companies means that they can just buy their own hardware. And run whatever models they can get their hands on in there.
Even if those models are Chinese. They (the companies running them on their own hardware) would be in full control then.
if the chinese models have this aura of inferior quality around them, then sure.
if they manage to frame it instead as open source, sovereign AI, the calculus changes.
But I don’t think this will be the first time in history that money doesn’t win.
Models with tool access (and internet-connected tools like curl) can, though… is this really a useful distinction?
People are naturally sticky. If they're satisfied with a product, they'll stick around unless something comes along that is so much better it gets people talking. Open AI's 1B+ customers won't give a shit about Chinese models. Enterprise customers won't move to Chinese models, especially just to save money. They are a non issue, especially if the M.O is just going to be playing catch-up.
Google, the titan mega-corp with more money than they know what to do with and a billion+ android devices to ship gemini on and it's made no dent for ChatGPT's consumer growth and adoption. Yet some plucky open source chinese models will?. It takes more than that.
How many people use an iPhone to its maximum? Isn't it the same with models? I think engineer's like to flex that they need a top of the line frontier model for their shitty corp projects. A Chinese model will write your Jenkinsfile just fine.
But also, I don't think you should use individual consumer behavior as a proxy for how the overall LLM market behaves. Current estimates are that individual users represent a pretty small share of overall genai revenue. Most of it comes from enterprise contracts. Those might be more price sensitive because the CFO gets a say in which vendor they choose.
For a couple years now I've been accustomed to choice of foundation model being a heavily scrutinized design decision because it's just about the easiest way to manage COGS. I can certainly imagine scenarios where, given the option to pay 1/2 as much in inference costs but with a 5% reduction in output quality, the boss would say, "Nah, that's a bad deal." But I have yet to encounter one in the wild.
Okay fine, it's $1,400 for a latest iPhone Pro with 512gb of storage. Point still stands.
> Most of it comes from enterprise contracts. Those might be more price sensitive because the CFO gets a say in which vendor they choose.
I think enterprise contracts are even less sensitive to price than consumer. That's the whole idea behind enterprise pricing. You'll see companies charge multiple times as much just for SSO for instance.
There are limits to this obviously, but to think that major commercial enterprises will switch over to some Chinese models to save some money is laughable.
So you switched away from the specialty product, but you're still comparing a top of the line 6.3 inch iPhone to a mid-market Android phone. Why not compare it to a Pixel 11 Pro with 512GB. That way they're at least comparable devices. But also, that would cost $1,220, which I suppose doesn't go well with the intent of suggesting iPhones are several times as expensive as Android phones.
The reason why I'm digging in on this is, when you start your argument with such a transparently silly comparison, it gives the impression that you're dealing in superlatives and willing to say just any old thing. Which undermines your own credibility.
So, getting back to corporate - multiple times how much for SSO? Okta Essentials is $17/user/month, which is a little over half as much as the base price for Cursor Pro, and about 2/3 the price of a basic ChatGPT seat.
Comparing two devices at the highest price point is besides the point.
For AI? Prompt comes in, tokens come out. The value of the 'ecosystem' is roughly zero to me. Switching costs between models are essentially zero. The UI and UX is the same whichever LLM runs underneath.
If you do use llm some, then the $20 tier makes sense, but that's not where the money is. If you use it a lot then a $100 sub probably makes sense. But if you're a BigCo that heavily uses it and pay API prices and suddenly you pay $200 million a month, well, maybe it's worth buying a bunch of hardware and stick it in a basement somewhere, and you won't upload all your secrets to your future competitor to boot. If US labs didn't care about the competition from Chinese models they wouldn't lobby so hard for banning the evil, unaligned, super dangerous communist models that want to steal our freedoms and make our gas expensive.
Ultra-cheap models like Deepseek are also very nice if you need to burn billions and billions of tokens.
It’s not unusual for someone to spend $5k in a week. 2 guys at my place just spend $150,000 over 3 months building out a factory.
We are aggressively exploring open weight models now.
Sounds like this will be another one of those things where everyone makes their money and it then will eventually become something experienced people 'blog' about.
I don't doubt it.
> Sounds like this will be another one of those things where everyone makes their money and it then will eventually become something experienced people 'blog' about.
Or not. The problem is that right now you and a whole bunch of others are trying to outrun a pack of lions. The lions have already taken down a couple of your group. The people further ahead have seen what the lions can do and they are attempting to speed up to the point that they can outrun them. Whether that will work or not depends to a large degree on what the other people are doing and they are not going to make it easier for them to keep pace.
That partially explains the lack of deep insight blog posts, besides everybody is just too bloody busy.
But you can also have many engineers operating it, testing outputs like you would in a real factory, etc.
Will be interesting to see how this pattern evolves. If it even makes sense.
If that were true, they would cancel subs and save themselves the load.
Yet that hasn't happened. So they must care to some extent.
With that, there are at least three distinct reasons to care about subs: PR, hook and training data.
Wait until you are forced onto an enterprise plan...
LLMs don't have that. For at least 90% of users, the model in the back end is completely irrelevant to them. Switching to Kimi won't lose them their iTunes music.
It’s pure genius. There’s a lot of bozo’s here who cannot appreciate it and cry about fairness.
Many here don’t understand corporate finance do they? Hence the marketing hopium to fool people.
How many Astroturfers and Anthropic employees are here to convince us otherwise?