upvote
> token usage

It is not exactly accurate, because it ignores speculative decoding which can often save a lot of time, compute and energy.

See this branch: https://news.ycombinator.com/item?id=50441444#50497854

reply
Speculative decoding will be model/software based and apply to all languages. The time, compute, and energy will be less compared to other models, but for the same model, hardware, and software used my reasoning still applies -- comparing English to other languages (unless optimized by the tokenizer) then English will consume less tokens and thus less energy. That is, it is still a function of the tokens used.
reply
So basically, the tokenizer is optimized for English and performs worse on other languages?
reply
roughly the tokenizer is optimized such that the training data (for the tokenizer) can be represented with the smallest number of tokens. and yes, English tends to be overrepresented in training data.
reply