I've done a couple side by sides on web chat with the same prompt on Opus 4.6, 4.7, and 4.8 and the output gets longer/more verbose on version increment. The enerr variants are definitely much wordier.
replyOn the other hand, the newer variants also tend to benchmark higher so it's not quite a clean argument of "hey the new version eats more tokens"
I think both things can be true: new models benchmark higher and eat more tokens.
replyFrom my experience new models are slower and use more tokens even on questions which gpt 4 answered correctly. It is mostly because newer models tend to be more verbose (even with prompt requesting short answers).
replyI bailed on Anthropic the moment they started blocking alternative harnesses like pi on their subscription plans.
replyIf I were anthropic I’d force that too. They offer the harness and if they control the entire pipeline then they can optimize the entire experience. It doesn’t have to be nefarious.
replyBut they gave us double the tokens! Then a limited time more usage! Then even more tokens "off peak" times! Then some new model released but apparently it inherently used 1.69x tokens! Then Fable is here but "it uses much more usage". But only until ~~the US banned it~~ ~~7th July~~ ~~19th July~~ who even knows.
replyAt this point I think Dario is just in his wellness retreat adjusting a revenue/profit dial.
Seems unlikely they'd be this dumb. The way to get us to use more tokens is to make those tokens more useful, not less. Anthropic is full of people (including higher-ups) who know this.
replyBut it is much much simpler to make it consume more tokens.
replyIt’s like that saying “What Andy giveth, Bill taketh away”, but in this case it is one company.
There is definitely a conflict of interest.
now reealize that LLMs are trained to produce tokens and like the halting problem, cant be trained not to produce tokens and youll realiE the AI labs are the perfect essential capitalist and like cancer, will keep growing useless tokens until it kills its host.
replyno amount of alignment will stop aomeone drom just shutting up.