Seconding this. 2-bit quant of DSv4Flash 0731 (Unsloth) running on lemonade-llamacpp on a Corsair Strix Halo. OpenCode had an issue giving me 400s constantly for the past few weeks, so I just went all-in on local dev with omp. Yes, it takes time, but also: it works quite well. I've since gotten OpenCode to behave, so I'm back to using that quite a bit, but the local machine is really good as a "smol" model in omp, for example.
reply