I wonder how well it performs in practice, because I can't help seriously doubting those benchmarks. That would put this 6GB model in Opus 4.6+ ballpark. Granted, that's mostly to Qwen 3.8's credit, but it's hard to believe that Qwen's already unbelievable capability density can still be compressed this much more.

That's correct. I think there can certainly be issues even with 4bpw with naive quantization (IE you'll notice far better results from a QAT 4bpw vs a naive 4bpw).

One such method that I've been meaning to look into further is Tencent's AngelSlim QAT/PTQ approach. They did a Hy4 preview release thats an STQ_1_0 at 2.38 bpw:

https://huggingface.co/AngelSlim/Hy4-preview-GGUF https://arxiv.org/abs/2602.21233

Of course, it's still 213g of VRAM I'd need so it's somewhat out of the range of what I can run locally. In contrast, this new Bonsai is nice because the original was already exciting for making use of low VRAM devices. Could breath new life into some of the older GPUs that were previously close to top of the line just quite VRAM constrained by modern standards and still quite cost effective for now.

Could've been better if GGUF implemented QTIP format. GGUF representation is a major limitation for llama.cpp quantization performance