I run DSv4 Flash at home on 2 DGX Sparks and am pretty sure there is no more cost effective way for me to do so. I'm not interested in running smaller models.
replyWhat's the tokens/sec you're getting on that setup (genuinely curious because it's a setup I haven't actually run myself)?
replyUsing regular DeepSeek-V4-Flash, I see 2000tok/s prompt processing and from 40 to 50 tok/s generation. Performance doesn't drop much at long contexts, DSv4 is really nice for that.
replyWith DeepSeek-V4-Flash-DSpark (Deepseek's new speculative decoding scheme), which is still barely supported anywhere, we're seeing a more steady 45-55 tok/s with bursts into the 60s.
> from 40 to 50 tok/s generation
replyThat's actually much better than I would have thought!
Thanks for the answer and it does make this approach make more sense as a budget solution to running larger models locally.