Running DeepSeek V4 without extreme quantization locally requires a lot of hardware.
replyThe IQ2 quants that fit into 128GB machines are very degraded.
That is true, it is a 1.6T parameters model so it requires a great deal of memory. I also heard there's a 2bit quantization that works well on Apple metal.
replyFrom what I read, ds v4 is very close with opus 4.6 performance.
replyThe full model is, not the quantized versions.
replyyeah that goes without saying. how can openweight, quantized version beat SOTA :)
reply