(www.businessinsider.com)
If virtually any other company in the world said "we have internal AI servers and we let our employees use them" ... the company would measure that as they would any other internal expenditure: by how much it costs them. You blow $50k a year on servers running your LLMs? It cost you $50k.
Only a company like Anthropic, desperate to make news and help their IPO, would measure by the cost they charge customers. It's as ridiculous as Microsoft counting every employee's use of MS Word ... as if it had paid the full consumer price of each copy.
A clear example is Google. They can easily give all compute to Deepmind, yet majority is allocated to be sold at market rate.
So, every compute Deepmind and Googlers get is lost revenue opportunity and must be measured as such.
No other internal tools or servers has this much external demand to be comparable.
Also, Anthropic is showing that to operate at Anthropic level, you need $300,000 compute spend per employee
They have infinite money to burn so they aren't going to optimize.
I've seen that trivial changes to workflows can token usage by 30%. Repeat this a few times and my token usage has plummeted over the last few months.
Only outputting test failures instead of having agents read test run output in real time. Not having build logs get fed into context. Projects like UCL (unified context layer) so agents don't rediscover the same facts about a code base again and again.
Giving tools agent output modes so they only print out the minimal info needed for an agent to get work done.
When I hear a company has an absurd AI spend I assume they aren't even trying to reign it in.
It’s not opportunity cost. It would be opportunity cost if their employees not using it would lead to additional revenue. However, if Inwas to go to Anthropic and double my token usage Anthropic wouldn’t decline my money. If Anthropic’s employees would stop using Claude, their revenues wouldn’t increase by an equivalent amount.
Their revenues are not currently supply constrained.
It’s just a cost cost. If their employees stopped using these services they would simply need less infrastructure, and therefore save on the cost to build and run that infrastructure. It wouldn’t translate into an increase in revenue.
I don’t agree. This is determined by elasticity of demand. If they have more compute to offer, that means they could offer higher subscription limits, more features like Duo to more people, better API pricing, and better enterprise contracts. This leads to some degree of increased consumption and more revenue. The degree is determined by how responsive customers are to pricing and features and performance. It’s not zero, so there is some amount that revenue would increase. We can debate the coefficient and the amount of compute we’re talking about here, but the premise is correct.
In order to correctly make that decision, they must use the opportunity cost of selling that token. I'm pushing back at OPs claim that Anthropic must use internal cost to calculate that trade-off. They absolutely should use external selling price.
you could not be more wrong, you cannot IPO if you say product is good enough. The whole point, the bet why its worth trillions is because they will reach AGI and "singularity" first, so the gap vs open source will only increase. If you say product is great as is, then you will be caught up and game over
They are doing the calculation on if the $150K/person is worth it, and it probably is.
Their revenues have exploded, they are unit-profitable, they need to also invest in the future.
AI infrastructure is supply constrained, so you have to make a decision: how much do I allocate to training vs inference?
Do I use it all for inference and do no further training? If so, what risks does that expose me to?
It's supply constrained in terms of global hardware production capacity, but that's not how it works for any given company. If Anthropic wanted more GPUs for customers paying them enough money to outbid the next guy in the market for the finite number of GPUs then they would do that and get a higher percentage of the finite number of GPUs.
So the cost of using more internally is still their own cost, i.e. the price of being the high bidder on more compute resources, not the retail price they charge to customers.
Yep. I bought a $20 per month Claude subscription last week. As I was using it last night I saw some tags in the log with prices at $36.xx and slowly rising. After asking and searching I got convinced that those are equivalent prices for API access at full price and Im not getting billed. As soon as $20 per month goes to $30+ per week I will change my behavior to lower it, up to canceling if it won't come down enough.
Unfortunately I keep thinking of more uses for it. Fortunately I don't need to do any of them.
That's API-list prices. They have an 80% margin over COGS.
Cost per equivalent output is dropping 10x every 18 months -> so in 3-ish years, if the trend continues, that will be $3k per employee.
Not bad. A lot of companies are already spending about that much or more on IT per employee.
Especially since that $3k bakes in an 80% margin. It's about $600 per year if you DIY at scale - which is about $50 per month - but at the $3k price, hardly anyone is going to.
In the West old-age dependency ratio (retirees per working-age adult) is climbing roughly 0.5 to 0.7 percentage points per year.
We've got to get 0.7% more productive for employees to keep the same quality of life (unless retirees get screwed, and since they vote and make up a huge portion of the voting block, I wouldn't count on that happening).
Further, more and more of the population has to transition into into elder care.
Unless we hit 3%+ GDP growth, us workers are probably going to see declines in quality of life (and we're probably not going to get those levels of growth)...
You could also look at the opportunity cost of not letting Deepmind do quite as much research. How much is additional research worth on the margin? There will be diminishing returns.
Thats a problem since they will always pay a pemium for compute. The data centers owners can compete on price if they choose to.
Bank reserves, Compute etc are trade-off decisions. Buffett vs Mayoshi Anthropic vs Google are being played out in real-time for us. Pretty soon you'll know which trade-off was better.
source? The most optimistic calculations don't come anywhere close to that.
They have never claimed that the API rates are the "real cost" of the compute spend.
I don't know where you got that from but the only time I maxed my $20 plan for a week I made the calculation that I had spend close to $200 in a single week. So maxing a $200 plan for a month should give you $8000 worth of token at API price. Obviously it's not $300k but it's still very far from the retail price.
$200 of usage on a $20/mo plan is a 10X multiplier.
The $200/month plan has 10X the usage of the $20/month plan.
So $200 * 10X = $2000/month. I don't know how you got to $8000.
Recent analysis by industry analysts like Semi Accurate have measured Sol 6.1 on the $200/month plan to be around $2000 of API equivalent use.
FYI there are 4 weeks in a month…
> Recent analysis by industry analysts like Semi Accurate have measured Sol 6.1 on the $200/month plan to be around $2000 of API equivalent use.
Sol isn't an Anthropic product though so maybe OpenAI is less generous than the former, I can't tell.
You have confused necessary and sufficient.
Maybe, but it also reduces supply that can drive up prices
There are a bunch of open source harnesses that are competitive with CC and they aren't using anywhere near that many tokens to do it.
Anthropic recently handed out free cloud resources (it's still free, guess they want you to get used to the fact that cloud is not free). I went through $250 cloud budget in less than 2 days.
Feels almost as expensive as LLM calls.
However, so is my phone!
Even within a single moment, if there is capacity for an internal request, just send it, if market traffic is currently at 100%, don't.
This is NOT the definition of opportunity cost. Please do not throw in random terms you don't understand. Opportunity cost is the value of the next best alternative you give up whenever you make a choice. WTF does it have to do with mis-classifying an expense?
Anthropic isn’t booking it as GAAP revenue, they’re just using the number for marketing.
If Anthropic had revealed their internal cost, there comments would be complaining that they were hiding something and giving useless, unrelatable data.
The expenditure is most interesting when compared to retail rates, because that's the rate the rest of us have to compare against.
Those individual subscription plans you get as a consumer are not what their big enterprise customers are using.
Those plans are cheap so you, as an individual, can get familiar with the product and convince your company to get it at work. Your company pays the API prices to get enterprise features.
Unless you managed to talk them into that, in which case, share your secrets!
As an example, as I understand it, if a hotel chain allows an employee to stay in one of their hotels for free when they go on vacation, the tax value of that benefit is what they would have charged someone for that hotel room, not how much it increased the hotel's costs to have one more hotel room full — which is generally almost nothing.
Of course, the IRS does agree with you when it comes to expenditures that are for the employer's benefit, rather than the employee's. If it didn't, almost all companies would have negative profits from the IRS's point of view.
Suppose a company could transfer an asset to an employee and have its value set as the employer's cost. A company could make a bunch of volatile investments and see which ones pay off. Then the ones that go south get sold back into the market as a tax loss and the ones that do well get transferred as compensation to employees at the purchase price instead of the current value, reducing the tax liability of both the employer (no taxable gain) and the employee (less taxable income). What would happen if any employer could do that with any employee and any asset?
Conversely, there is a separate carve out for "employee discounts", which is essentially the employer offering its employees its own products for something in the neighborhood of its own cost. Why? It's not in any way a principled accounting distinction, it's because it's harder to use for tax avoidance at scale, because then a soap company can give its employees a discount on soap, but people only need a finite amount of soap and then the soap company can't use it for cars and real estate and stocks.
If what you said was how taxation worked, a hand tool manufactuting company giving hammers they make for $1 and retailing for $2 to workers building a shed on their property would be increasing the workers' income tax bill by $2. Of course, in reality the workers get no income tax bill on the hammer given to them for work and the company can get a $1 tax writeoff from the opex of making the hammer.
The extremely discounted internal cost promote waste, so there is always a room for improving efficiency, but that too has a cost.
In a similar way, internal employees would have also a "standard daily rate" like an external employee.
Plus electricity
Yea, isn't the entire point of RSI to replace your employees altogether? So you'd spend an infinite amount more on AI than employees in that theoretical.
Like, if you were in an energy generation business spending way more on fuel and capital expenditures is way better than having zillions of employees earning checks , it would be a sign of efficiency.
So you are apparently now claiming my local train company or law firm will be using as many tokens per employee as an A.I. lab.
This is as if you learned that Anthropic scanned 1000 books per employee this year to train their models and your conclusion was that the best law firm will be scanning 1000 books per employee by 2029.
This is the world of spend more, get valued more; obviously the investment will be used to spend more and get more, so it must be good. This is investor-think. If you don't have spend, why do you need investment?
(Or it could be bad. Very very bad)
>I wonder how much dog shit has crept into their codebase by this point.
You'd think they'd know not to shit where they eat… ?
Given the speed at which Anthropic regularly pushes updates (eg their ridiculous app versioning that's in the thousands for v4 now) I doubt they're doing anything that looks like quality processes, even AI-integrated ones other than basic code review skills.
I imagine it's as much of a mess internally as it is with any other company whose execs are concerned more with bottom-line valuation rather than maintainability, extensibility, and sustainability. Just lots of spaghetti-at-the-wall hoping that something sticks long enough to drive up the IPO.
On the flip side, software I used that was majority built by 'real human engineers' coding by hand is utter garbage.
As a recent example, Facebook Ad Manager is extremely slow, plagued with UX issues, blatant state management problems (double scrollbars, iframes, random rerendering).
Other examples of 'real engineers' building software include SAP, Oracle ERPs, most banking software, most government websites. They're trash.
Human engineering is the reason your personal information gets leaked every other week. Human engineering is the reason there's a critical vulnerability in core software every day at this point.
The average software engineer cannot build good and secure software. Just like the average car driver is a safety disaster on the roads.
Mythos at the highest thinking level is not the only product from Anthropic though.
I guess kinda put you in a bubble a little, I wonder how the experience of Anthropic eng who left the company would look like, when such eng would go and spend few $K a day in non AI lab, would that be encourage/allowed/taken as example?
Especially when I need to plan my next hike up Mount Shasta.
Now spend 0.5x their salary on AI, and they become 5x productive compared to no AI.
Now spend 1.1x their salary on AI, and they become 20x productive compared to no AI.
It's still obviously worth it to spend the 1.1x. But it is truly remarkable that the biggest cost of hiring a knowledge worker is no longer their salary, but their AI usage.
There is no implication here that employees are being replaced by AI spend, as long as all of the productivity is generating profit. The AI usage still needs a knowledge worker to direct it in ways that it can't direct itself, so you still need the employees.
This is what I am getting at. If this were true - and I've yet to see evidence it is, despite some anecdotal evidence (organizationally, IME, it's about 2-3x for power users, and probably a net loss of productivity in certain areas) - why would their headcount be growing so much still YoY, pretty much doubling every year. If the AI is 10-20xing why such a big jump in headcount? Yes, the numbers are shiny, the growth is apparent, I would just expect with this AI narrative they'd be "doing more with less" as this thing improves every year.
> The AI usage still needs a knowledge worker to direct it in ways that it can't direct itself, so you still need the employees.
This is the story AI sells though, and Anthropic has participated in it heavily. Yet, their interview process requires hard skills and doesn't let you use AI, so clearly, what you're saying is correct, it just doesn't really align in my head with the whole knowledge worker being replaced narrative.
That wouldn't really be an expected outcome for a company like Anthropic. They need to do the maximum they can with the funding they have, or risk losing out to competitors who don't take their foot off the gas.
But it's also true that when I'm working on 2 things at once, the bottleneck is me. I'm spending all my time giving Claude direction that it can't give itself.
So you basically have to separate out the tasks where AI can produce a 20x productivity boost, and the stuff where it can't help at all.
There's still so much stuff where it can't help at all, you need real human employees for that. That's why Anthropic is continuing to hire at breakneck pace.
And that's what I'm saying -- knowledge workers aren't being replaced. They're just accomplishing tons more work, and companies aren't running out of work to do.
And the product now iterate 2 times faster with 10 times more bugs and customers are pissed.
Looking at you Apple: https://aperocky.com/blog/post.html?slug=apple-watch-sucks
There's a lot to unpack about this statement.
Including the fact that it's probably not true.
If something is compensation, it can often be negotiated and traded off against other compensation components. How many people have successfully negotiated AI tokens away in favour of salary/bonus/stock? I'm going to guess "approximately zero".
I haven't seen anyone treating AI tokens as employee compensation. If anyone can show me an article where employees were paying taxes on their token use at work, please share it.
Companies have been treating AI tokens as part of the cost of having employees, like the cost of their office or travel expenses.
True, though that phrase understates the author's problem a bit, as it is surrounded with more wrong-words, indicating it's a wrong-idea rather than a slip of the tongue.
As you note, it's closer to office/travel expenses. Employee computers, licenses for software on those computers, licenses for SaaS stuff, paying for bandwidth, etc.
Many companies used to brag about about how nice the laptop or office was as a way of convincing people to work there. Often in the same breath as compensation.
But compensation is only one component of cost.
So I think this guy is confused about what it costs to have an employee and what the employee gets out of it. Possibly intentionally.
By giving hundreds of workers each a powerful desk-sized Alto minicomputer for their own needs, with a high-end graphical display, they gained insight into what desktop personal computing would look like 15 years later, enough insight to shape it profoundly. It's been 55 years, and although multitouch screens are proliferating, I'm still typing this on a keyboard hooked up to a framebuffer-driven display with a mouse hooked up to it, displaying hypertext, festooned with windows, icons, and a mouse that moves an onscreen pointer at the same 135° angle they settled on 55 years ago, positioning a vertical line to indicate the insertion point, with scrollbars. The display has twice as many pixels, the pixels are color, the mouse pointer is white rather than black, the screen is a flat LCD, the scrollbars are on the right, and the cursor blinks, but it's very recognizably descended from https://smalltalkzoo.computerhistory.org/HOPL-St78.html rather than, for example, ITS or GENESYS.
Only people with the luxury of unlimited zero-cost CPU cycles had a chance of seeing what the future looked like, and that allowed them to find out what worked and what didn't work in that environment. Nowadays, the regular workers who don't have the luxury of unlimited zero-cost tokens are manually reviewing voluminous AI code, because, even though that's a lot of slow work, they're still much faster programming that way than they were without it. Anthropic, instead, is doing insane things like eliciting a billion lines of Lean proofs to show that existing open-source libraries can't fail their OSS-FUZZ tests — and fixing all the bugs that effort turned up, in the process.
I know there's a point where companies get big enough, and Anthropic forces them over to the API, but presumably such companies negotiate heavy discounts instead of eating the 200x overnight price hike.
I doubt they are bringing that much value to the company for whatever work is done with that much token.
And I’m sure more than that. But there’s obviously plenty to work on, and they’re not at the point they can just Claude their way through it.
The real question is what could they have done with open weight models on the same compute they’re renting. If the answer is “basically the same” then this flips the entire conversion back to what is Anthropic’s moat here.
This is the nature of managed anything. AI efficiency will improve but essentially this is what automation does.
It’s not weird once you think about it. Money doesn’t create an operator. But operators can be cheaper than tools and often are.
I'm sure in several US-based companies, non-US workers AI spending is already beyond their own cash comp.
So it exceeds employee pay if you discount the vast majority of employee pay.
Would Anthropic themselves be paying this much if it was actually billed? It is not obvious.
Is Anthropic a good sample for AI use? Surely not, since they have special needs (dogfooding) and a vested interest in proving that AI usage is the future.
Can we compare their results pre and post AI usage? Close to definitionally impossible.
Are their employees good examples of AI-assisted engineers? Not really, as (one would hope) they're more well versed on the topic than an average eng.
- @fable, plz s0lv3 Riemann. You are doing great.
- @claude. rewrite yourself for the 10000th time. Use as many tokens as you like.
Or has it broken out of its jail and taken over the board? Dario seems like a bit of a windbag anyways, AI can probably automate most of what he does these days.
Correlation does not equal causation?
All this news says is Company A uses internally the resource they sell externally at ($X) value. It's not saying how much it cost the company to generate said consumed resource
These are not mutually exclusive points, and I find it a common issue to get both sides to understand.
both of those companies combined are smaller than the number of people microsoft is trying to buy out right now
That's just reverse naivity.