The opacity isn't accidental. When customers can't forecast cost, they can't build an internal business case to switch, which is more durable than lock-in from product quality. The first AI coding vendor to offer deterministic $/task pricing will own enterprise procurement conversations. Right now the entire market is hiding behind 'tokens' specifically to avoid that comparison.
replyAlso, certain models are more verbose than the others. We are basically at the mercy of a model who likes to ramble a lot.
replyim fiarly certain the knob on the machine that controls length of redundant comments and docblocks is cranked to 11. it makes me curious how much of their bottom line is driven by redundant comment output.
replyRelevant post: https://modal.com/blog/dollars-per-token-considered-harmful
reply(disclaimer: I work with the author)
I completely agree that requests are what should be charged for. But I think there are two things, given that requests aren't all going to cost the same amount:
reply1. Estimate free invoicing the requests and letting users figure it out after the fact. 2. Somehow estimating cost and telling users how much a request will cost.
We have 1, we want 2.
Like if cars measured fuel efficiency or range using the knobs in the tread on your tire.
reply