Just one of our analysts was burning around 30-40 million tokens per project.
replyEDIT: per project, not per month
EDIT 2: digging out an old bill from 8 months ago when we bought the server, $14k per month is what we were running and it was going up, a lot.
Anthropic is very good at either convincing or forcing (I’m not sure which) companies to pay for tokens rather than for subscriptions. My old job moved from subs to tokens and everyone started running out of budget mid month. It was a bad move for a company that laid off all of QA and 2/3 of Engineering and committed to everyone making up for it with LLMs.
replyI don't get this, why do people pay for API tokens at all? What is the reasoning?
replyI don't think Anthropic or OpenAI offer subscriptions to enterprises - it's a hobbyist/solo-entrepreneur kind of thing. I seem to recall SpaceXAI had some kind of hack going where they were trying to use subs with Anthropic and then got cut off.
replyEnterprises also need very stringent IP protections that you don't get through the subscriptions. Even the Anthropic-API per token license isn't great - sometimes you need to go to 3rd parties like AWS Bedrock to get guarantees regarding no multi-tenant GPUs.
You can have subscriptions up to 150 seats with Anthropic.
replyZero Data Retention is the big one I think (and also the ToS don't allow multiple personal accounts for businesses anyways)
replyYou are forbidden from the subscriptions on enterprise
replyThey don't have the choice to begin with
replyI have only ever paid for tokens and never used subscriptions because I need to know that I can generate sufficient real world value at the true cost of inference.
reply[dead]
replyOh boy... we are talking about insane token spends in corps at the moment. 100$ is a blip.
replyI think you should compare with api cost? I am using $200 plan and it is probably around $12k in token cost....
replyAre $12K worth of token guaranteed or the company may adjust token generation speeds?
replydeleted
reply