> Please don’t respond if you are speculating.

I doubt you are going to get a response from an anthropic employee, but I think it is safe to assume they have swapped to a new tokenizer because it improves the performance of their models.

> the reason for the newer apparently less efficient Anthropic token encoding

Less efficient in token usage but per the blogs; it enables the model to perform better.