"The pursuit of excellence does not need justification."

https://x.com/mitchellh/status/2074225453217505494

This depends on what your workflow is. There are use cases for tokenization that don't always involve immediately feeding the text into a model.

It's pretty funny but then again, why not if it's as trivial to simplify as it appears