I guess it depends if we're talking about decent new car at $40,000 or $95,000? I've seen lots of people claiming they're running v4.1 flash 1017 on local hardware but then it turns out it's something like the unsloth Q2 XXS or similar size. The Q8 is 508GB so assuming for a moment we're not running it on a Mac, we're looking at a cost of five or six 96GB GPUs to load the whole thing with a good sized full context and KV cache.
replyHaven't bothered with DS4.1 personally but I've seen reports of running it with native weights on 4x DGX Sparks. Currently ~$30k but last month that setup was ~$20k.
reply4x 96 GB runs the NVFP4 quants, which are indistinguishable from Q8 in real-world performance for this particular model. That's not always true but DeepSeek is unusually quantization-friendly. Also remember that a big chunk of the model consists of engrams that can live in system RAM or even on an SSD if necessary.
replyI imagine the various GGUFs are still pretty impressive, but haven't tried them myself.
> new car
reply...which is still tooo much for most people to spend on "AI hardware"....
More like, the ability to chain a demon in your basement that does what you tell it and can't be rug-pulled by his bosses in San Francisco.
replyAI is the endgame for the joke that computing is the act of etching arcane signs into crystal and imbuing them with lightning.
replyRecently saw a hand drawn cartoon to the effect of, a person is asking Claude or chatgpt to do something, it refuses, so the next step is "make the Chinese one do it", and the Chinese model is represented as a chained demon
replyJust gotta make sure that pentagram is rock-solid!
reply