Qwen3.8-Flash_Next should be in the running for that title as well. It's astonishingly good. I use both of them. If I need more memory on the machine for other things I run 27B, If I can get by with much less RAM I run QFN (takes about 102GB on my Strix Halo box with 262,144 context). QFN runs quite a bit faster on my hardware - about 50t/s. 27B with MTP starts out around 22t/s but degrades over time as the context fills up. Both are great models, but I'd give QFN a bit of an edge (at least for the coding projects I've worked on). But both feel on par with Claude Opus 4.6 - 4.8.
qwen was faster in t/s, but feels like glm spends less tokens thinking and seems better at finding issues, to the point where it ends up being faster.
Unfortunately it still hallucinates, which limits it's usefulness for real world tasks. :(
HF is telling me that IQ4-XS is the largest one likely to fit without much contortion.
https://joeldare.com/a-local-open-weight-model-builds-its-fi...
I think it can be made to work if you’re willing to tweak the prompts.
But, to be honest, I find my time to be worth more than the $20 I spend on a light weight frontier plan that can do this without the hand holding.
I want local to succeed, however, because of things like Anthropic stopping games based on other games.
Curious, why/how does that help? Or do you have some niche needs?
As such, if you want to replicate big model behaviour, you need to basically do context augumentation. How you do that is up to you for your own needs.
Why? Most existing harnesses let you point them at whatever API you want. I'm currently have a great time with Juggler.
How do you validate and verify that outcome?
Or does Qwen-3.8-27B do the validation and verification for you as well? If so, how do you ensure that's correct?
Personally Qwen3.8-27b has not proven itself to me for the projects I’ve tried to use it for. It doesn’t seem to have as much encoded knowledge about the native Mac development stack.
I don't think you meant to say this but it feels like you're saying "vibes".
It would be better to use a scoring framework like Harbor or similar?
I think over time we'll realize this is the way forward, you want the smartest model with the least amount of embedded knowledge and you provide the knowledge externally or merge it into the weights for the domain you're interested in. Then you don't have to train a ginormous model every 6 months to pull in new information.
A RAG solution would make time to first token awful.
in your specific workflows, where specifically is the time to first token a dealbreaker? what's your typical target and what's a "no go"?
The only reason Qwen is more popular is because it can run decently fast on MLX, while Gemma needs a discrete GPU with 24gb of ram to run at usable speeds.
Also, theoretically speaking, any small model can be as good as a big model if all they have to do is summarize text. Back before agents were a thing, we had prototype coders running of local inference models from Mistral for ai dev. The workflow was to generate llm summaries of the codebase, and a summary of changes to be made. Then we would grab search results from the web, feed it to an llm to summarize, and then finally combine all of that context into a final inference run which would generate code, then the codebase would be run automatically, any errors would be llm analyzed, and an llm would generate the next change to be made in the above cycle.
Nowdays, we can do that with pi coding agent and Gemma4:31b. But we still have plenty of custom harnesses for things.
I don't understand why megakernels haven't become the default choice. The speed advantage is crazy. I made a few changes to the openjet megakernel to run in a dual gpu (older gen, not blackwell) box and doubled my token generation speed on qwen3.8-27B (4-bit though).
I keep debating adding a third matching gpu to see if I can push the speed a bit more and move back to the 8-bit quant, but I'm stuck on the idea of just building a Blackwell gpu system despite the cost factor. Plus a third gpu is going to going to be weird for the tensor split and P2P data syncs and my workstation is old with 3rd gen PCIe etc etc.
What frontier models are you using? I'm literally sitting with Qwen 3.8 27B locally and trying my damnest to get it to work at least 80% of the quality Astra et al have, and can't get close to it. It keeps being so verbose, regardless of inference parameters, system prompts, reasoning effort, and keeps going on tangents. I'm running the BF16 weights via vLLM, can't imagine how much worse the experience must be with the quantized weights people run with the average consumer GPU.
Even with tight guiding, as I guess you could say I do more "AI pair-programming" than vibecoding, it just loses the plot to often and waste time, enough that it's clearly not an alternative to today's frontier models.
Local models can't get even close to frontier ones; it's just bragging. There's a reason why models like Astra have 2 orders of magnitude more parameters (and run on specialized clusters).
Compared to frontier models, local ones (like Qwen Flash) are much slower, they have much less room for context, and most importantly (people forget to mention) they produce much buggier code. Ah, Qwen also goes into loops sometimes. I've used them on powerful (consumer) hardware.
Having said, no doubt that Qwen-Flash-Next is considerably better than Qwen-3.8-27b - but on the other hand, it's several times bigger. Deepseek Flash also keeps getting better but bigger.
For comparison, I also used GLM-5.3, Muse-Spark etc, with no significant improvement for ordinary/BAU coding tasks.
99% of the time, I'm not. But if it cannot reasonably write a basic CLI for transforming data without properly following a ~1K token long system prompt to the letter, then waste my time on it?
"Ah yes, I forgot about part X" or "I misunderstood the intent of Y" are so frequent compared to SOTA models, that it's hard to justify relying on it, even though I'm able to run it pretty much 24/7 at home, if it just was intelligent enough to actually make sense to do so.
I did had it forget some stuff over long runs that went through several context compactions and i'd expect a larger model to be able to handle more vague instructions or have wider breadth of knowledge (if you're trying to make Qwen 3.8 27B do stuff outside the mainstream you need to provide it with some reference material to use) but i never had a problem with it not following instructions.
The harness might matter though but i'm using plain Pi (which is a very barebones harness with a very small system message) without any addons.
Would something like LMCache help here?