I use lemonade llama.cpp with a 2-bit quant of DeepseekV4F 0731 that runs at about 15 tokens a second. I use it as my "smol" model in omp, with default going to OpenCode hosted Deepseek (faster inference).

I haven't done work lately on Dwarf Star. I got it running a few weeks ago and had my agent make a couple of modifications to it to get it running, but it wasn't materially better than my lemonade setup. I might revisit after this post, though!

https://github.com/lemonade-sdk/llamacpp-rocm

I've switched from halogen to gufo a while ago: https://github.com/gufo-org/gufo It's OSS and I think it's faster.