Qwen3.8 27B (which I adore) is nowhere near Opus 4.8 at puzzle games testing fluid intelligence, https://quesma.com/blog/baba-is-aug-2026/
replyyeah it's more like opus 4.6 iirc
replyLately I've been throwing tasks at Qwen and a frontier or recently-frontier model (as well as Kimi, GLM, etc) and the smaller parameter models are not really comparable to Opus when it comes to making intelligent decisions about greyer areas of good software architecture.
replyAmazing results for open weight and that size, but a really long way off, and I'm extremely skeptical of benchmarks that show these smaller models as being anywhere close to Opus 4.8 (or even earlier Opus's).
I’ve been doing the same thing, giving the same tasks to Qwen 3.8 27B and Opus, and the main difference is that Qwen does not consider edge cases which Opus catches. It’s good at the happy path, but even when hinting that there are uncovered edge cases and gotchas it’s oblivious to it. So I feel like I need a bigger model to do planning/review.
replyTo be honest I’ll ask a model to specifically think of edge cases but I won’t expect any model to do the edge cases of its own volition
replyAs a counter to that - I've tried various flavors/quants/full weights and Qwen 3.8 27B has been entirely useless at anything non-trivial. Sure - it can do some boilerplate work (though, even armed with a well written spec and working within a very well known framework it went off the rails and did things in a way that were... um... questionable at best) but I don't see it as anything more than a personal assistant style model. Zero chance I'd "work" with it, I spent days trying to get it to do something for me that was usable that I didn't have to have reviewed and refined by a frontier level model or myself. Couldn't do it. The idea that qwen 3.8 27b is _anywhere near_ Opus 4.8 is laughable. Pure benchmaxxing.
replyDS4 Flash 0731, on the other hand, wildly opposite experience. Would recommend.
GLM 5.2 - even quanted down to a hybrid 4/3 bit setup is amazing for everything but the hardest/most complex stuff in the same projects/realm.
I've had the exact opposite experience. I've been using 3.8 for my daily driver since last week, and I've gradually been giving it more and more complex tasks as it continues to deliver high quality results. Now I am basically handing off large complex features, and 3.8 is doing the planning, task breakdown, implementation and review with just a few notes from my side.
replyThe tradeoff is time (especially on RDMA4 hardware) - it does take a long time and spend a lot of tokens to get to the result, but I've found I can trust the results enough that I can queue a lot of work, essentially have it running all the time and achieve a decent velocity.
It's the first small local model I've felt like I can do real work with.
I can't get 3.8 to exit thinking loops. It will just think and think and think on the most trivial topics. I wanted it to port a speed test powershell script to c#. Claude opus 5 completes it under 60 seconds. I let 3.8 churn about 6 different times for 30+ minutes and it never wrote a single line of code to disk. It wrote lots of lines in thinking.
replyunsloth/Qwen3.8-27B-GGUF UD-Q3_K_XL DSH (pi)
Any tips?
That sounds like something is off - I'm using UD-Q4_K_XL on pi with xhigh thinking, and unless I'm vastly underestimating the complexity of the script that's the kind of task I would expect to take a couple of minutes (getting ~30t/s decode). What server are you running, and are you using the recommended parameters from qwen/unsloth?
replyset thinking to minimal and use these jinja templates: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
replywe went from 62% completion to 92% using a claude code harness
3.8 doesn't have a minimal thinking mode, only low, medium and xhigh.
replyWhat harness? I've had similar results as _Implicated_ said above - it's not done well in any of the tests I've tried with it. I currently have it hung off DS4Flash as a pseudo-vision tool and subagent only because of this.
replyI'm using pi inside a self-made harness. I've found going super lightweight with context (AGENTS.md is maybe 20 lines) and letting the model discover what it needs to gives the best results.
replyFascinating that after years of for me this for me that, literally no one writes 2-3 other words like “i do react frontend” or whatever just for us to know why the results are different
replyThey also need to tell people what quantisation they're using. Because some 4 bit version is not the same as BF16, no matter what KLD suggests.
replylol, exactly.
replyYour assumption is incorrect.
replyBoth of you should mention what quant you're using. And as another comment said, what tasks you're doing, i.e. coding, classification, summarizing etc.
replyI gave Qwen 3.8 27B and Opus 4.8 the same task in the same codebase. They both came up with the same diff. It wasn't a particularly challenging task (removing a feature flag and updating applicable specs), but it was character for character.
replyWow that's uncanny.
replyGiven it was probably one of the simplest things you could change in the codebase, the kind of stuff you give a new developer on the project, I'm not sure it's so telling, there is usually just about one way to remove a feature flag.
replyThe specs that were written are what I found surprising, not that it removed the flag in the same way.
reply[dead]
reply