For example on programming, terminal tasks, troubleshooting etc Qwen3.8-Flash-Next is almost as good as opus 4.8. On "knowing what it doesn't know" it is far behind opus, but leads all other OS models I tested by a wide margin.
It out classes Glm5.3 flash and leaves models like DeepSeek 4.1 far in the dust. But you still get those silly aggregate benchmarks where GLM and DeepSeek are higher.
It is a small model, only 120B anyone that expects it to excel on general knowledge or to have a bajillion ready made workflows memorised is kidding themselves.
Everyone knows the bigger the model the better it will do on raw content of knowledge. But you know what compensates for lack of it? A good harness. But there is no compensating for lack of ability.
So in short I find all those aggregate benchmarks useless.
The raw problem solving capabilities of Chinese open weights models are still weaker than or on par with Opus 4.5 but they have been post-trained to be much better at agentic coding, cybersec, and computer use.
And that is okay. GLM-5.x and DeepSeekV4.x have both been highly useful on their own.
This is 100M fewer than Opus 5.5, 40M fewer than Fable 5.1, and about a third of Sonnet 5.5. In the top-6, it's 3rd, beaten by Muse Wonder at 140M and Astra at, impressively, just 60M.
Opus 5.5 high trails K3.1 by 1 point, but outputs only 53M tokens, so it indeed comes out cheaper per-task. I think it's still remarkable that this is effectively an open model that matches the frontier for the first time in a long time, pretty much since DeepSeek R1.
> reasonably priced when comparing to other models of similar price
???
Very similar to my adversarial programming language (KillSwitch) benchmark[1], at least on pass rate, it is much slower though.
From my own use of the models I find their ranking to be totally inaccurate.
The per token charts do not provide meaningful information for me
I also find the provider benchmarks fascinating: https://artificialanalysis.ai/models/deepseek-v4-1-flash-101...
I think lm-arena is a better concept that reflects the actual target users, but it only captures chat, and relies on users spending their time improving it.
I do trust epoch more than AA due to capital incentives, but in this case I don't think like there is a difference
It's fine if you don't trust them. You are free to put together your own benchmarks, run your own tests across models, and objectively compare results.
That's how you gathered your data to support your claim other benchmarks are totally inaccurate, right? Right?
So their base price is 1/5th of Opus 5.5 per token, but if you want "highspeed" it's 2x that and then I guess it generates 2x the number of tokens as well?
That might suggest that Opus is competitively priced after all.
But as noted by other replies, it's still lower than Opus/Sonnet.
Currently for today, it's 1.19T input tokens, 5.41B reasoning tokens, and 4.29B completion.
Therefore, reasoning + completion is 0.8% of the total usage. Actually below my estimate!