It's not gonna stay that way. Token cost is being massively subsidized right now. Prices will have to start increasing at some point.
replyThis is hard to say definitively. The new Nvidia Vera Rubin chips are 35-50x more efficient on a FLOPS/ megawatt basis. TPU/ ASICS/ AMD chips are making similar less dramatic strides.
replySo a service ran at a loss now could be high margin on new chips in a year. We also don’t really know that they are losing money on the 200/ month subscriptions just that they are compute constrained.
If prices increase might be because of a supply crunch than due to unit economics.
Given the massive costs on training, R&D, and infrastructure build out in addition to the fact that both Anthropic and OpenAI are burning money as quickly as they can raise it, the safe bet is on costs going up.
replySeems like the real costs and numbers are very hidden right now. It’s all private companies and secret info how much anything costs and if anything is profitable.
replySome say margins could be up to 90% on API inference. The house always wins?
replyThat's like saying driving for Uber is profitable if you only take into consideration gas mileage but ignore car maintenance, payments, insurance, and all the other costs associated with owning a car.
replySome could say anything when there’s no proof.
reply