I have a self hosted Qwen 3.8 27B and I find it unbelievably cracked and dedicated. It's at least credibly attempted everything I've thrown at it. Just today I had it write a toy compiler with a JIT backend just to test out a concept, and that was with 4-bit quantization and 8-bit KV cache. Something has to be going wrong with your deployment.

Check sampling parameters and chat template, make sure you have adequate context window, turn reasoning effort down. It should be able to one shot a small app without intervention.

I run it locally at q4_k_xl on a r9700 with kv cache bf16 and while it thinks a lot, it’s still fast enough to do the task.

This model had its knowledge replaced with reasoning ability. The chain of thought what makes this reasoning effective.

So this is why you need to let it think and don’t quantize the kv cache.

I want a Qwen 3.8 27B hosted locally but I don't quite have the RAM for it. And, I don't want to buy the RAM until I prove I can use it.

Yesterday I did have success with Gemma-4-12b with 128k context. It fits in my RAM and it's relatively fast on my hardware.

I had to give it prompts that are quite a bit different from the way I use foundation models, but I did get it to work quite well. I feel like I could learn it's differences and get good at using it for real work.