upvote
I think we don't have a good draft model for better speculative decoding yet (e.g. DFlash 2). Once we do, it will be faster.
reply
It very well could be faster, but right now it isn’t.
reply