Deepseek's DSpark does dynamically adjust speculated token count per user/completion.
replyhttps://arxiv.org/abs/2607.05147
But they do so to maximise total throughput, I don't think there's reason to do that for batch=1.
Don't you just need to occasionally try n+1 for your current setting? Then record acceptance rates. The nature of the thing is that if you try n, you already have a good sampling of how well m works for all m<n, right?
replyThis is all a pretty active area of research, both adaptive drafting, different draft approaches (DFlash etc), I expect to see really big improvements land in the popular runtimes this year.
reply