So they move to non SOTA models.

Or get multiple teams. Just use git instead of artifacts ;-)

API token inference is subsidizing the rest of the business.

People keep saying this, but inference itself is not what is being subsidized. The expense for electricity/hardware/datacenter operations/etc for an inference token is less than what they charge for it, so that cost doesn’t HAVE to go up to be profitable.

The thing that is being subsidized is the TRAINING costs, and that is because they aren’t selling enough tokens yet to pay for the training costs. This is just like most other growth businesses, though, and if enough people use enough inference, it can be self sustaining.

Price per token is dropping (though anthropic are doing dastardly things such as using new tokenizers that use 30% more tokens), efficiency will likely increase.

If the subscriptions (of all of the big companies - unlikely?) give a poor amount of value in the future, will move to trusted/open router and use a harness that uses the latest cheap good models for most things, more expensive models as orchestrators.

Already there are non-SOTA models that should be sufficient for most non-coding tasks, and soon for coding, also.

The future is bright, in that respect, without subsidizing the ridiculous IPOs.