It looks like 2 bit. I'm really struggling to understand why this got so many upvotes. Colibri and others already covered this ground months ago.
replyYeah - at 2bit the accuracy must be pretty poor?
replyVibes from localLLAmA suggest a larger model (e.g. DSF) at a low quant still beats a smaller model (e.g. qwen 27B) at a higher quant.
replyWhere that threshold is crossed is up for debate.
Also: a heavily quantised recent model beats its full precision older versions.
replyQwen 3.8 27B Q3 readily beats Qwen 3.6 27B FP16. Difference in hardware requirements and token rate is of course massive.