upvote
For values of "usable" that include "14.6 seconds/token". It's a cool accomplishment! And newer hardware would speed it up some. But I think I'd want something a bit faster before declaring it usable in practice.
reply
14 seconds per token? Not tokens per second. Seconds per token? That’s nowhere near the cusp!
reply