> they make a ton of money on inference
replySo it is stated, but is it actually true? I am not convinced.
Besides, it's not as if they can suddenly stop training models, the moment you do that you've spelled a death sentence for profitablity because Google and open source will very quickly undercut a 15 year break even timeline.
Agreed, the revenues are big.. but very small next to the datacenter bills.. even if a fraction of which are being used for inference, it's hard to argue they even break even. That's before all the other costs (Super Bowl ads, billions in compensation).
replyIt's widely reported and acknowledged as true.
replyWell, the only people with any ability to acknowledge it have a massive incentive to do so, and I've been around the block enough times to know that startups will use every trick in the book to paint a rosy financial picture, even when it's extremely misleading or occasionally just straight up lies. In the current climate of AI hype my skepticism is even greater.
replyI'll believe it when I see it.
Where and by who? Critical context missing here.
replyDario Amodei @ 18:05
replyThe CEO hyping his product and the viability of his business during an interview with Stripe does not, at least to me, qualify as “widely reported and acknowledged”
reply[dead]
replyfrom what i understand, the issue with inference is it doesn't scale as user count grows the way traditional saas scales. In typical saas adding users requires very little additional capacity. However with inference, supporting more users requires much more capacity to be added. I don't know if it's quite linear but it certainly requires more infrastructure to support additional LLM users than say a web application.
replyAnd the existing infrastructure routinely struggles for several of the well known players. You can literally tell when it's getting bogged down by workload. And that's after all the absurdly large datacenters we've already established at significant expense (to both the corporations and the average person).
replyAfaik Anthropic still loses money for their main product in this space: Claude Code and their Max plans.
replyThis became immediately clear to me over the weekend when I used Opus via API key. I had it review the code for my (relatively small) personal blog to create an AGENTS.MD - it cost me $3.26.
replysame here... The API costs are absolutely insane for any real usage. This is either high prices to make sure no profitable competitor to claude workspace or other agent system emerges, or heavily sponsoring of their own soluions.
replyApi cost need not correlate with running cost.
replyNot really. They are burning money on hardware, resources and payroll without meaningful return prospects.
reply