Apparently Qwen 4 might land this week, and it uses the same architecture as 3.8-Flash-Next (so various LLM serving tools should be able to handle it already). I'm hoping Qwen 4 includes some smaller models - the last 4B and 9B from Qwen were for 3.5.

Agree, I run QFN on my AI Pro R9700 (32GB) using https://github.com/Niko1221/Strata It's even faster than on a Strix Halo. Typically I get around 90 tokens/sec with a small context, down to 70 tokens/sec when it's near the maximum context usage.

QFN is obscene. It runs faster than 27B on all my setups, and it always outdoes it in my uses (from coding to translation to vision-related): from somewhat to a lot.

Same experience, the 4bit quantized xlarge flash next runs at around 70t/s across 3 radeon v620s. Still have to retry the MTP version with full resolution.

I switched back to glm-5.3-flash from qwen3.8-flash-next.

qwen was faster in t/s, but feels like glm spends less tokens thinking and seems better at finding issues, to the point where it ends up being faster.

> Qwen3.8-Flash_Next should be in the running for that title as well.

Unfortunately it still hallucinates, which limits it's usefulness for real world tasks. :(

Which quant of QFN are you running on your Strix Halo?

HF is telling me that IQ4-XS is the largest one likely to fit without much contortion.