On current GPU hardware/firmware... The cynic in me imagines in the future GPU vendors wanting to implement only loading signed weights to your card, aligned with your "subscription level", for your "safety".
This is why it's so important to have more than one hardware vendor. The bigger the ecosystem, the harder it is to cartel. I cant believe it, but i am pinning my hopes on china to break through with commoditized chips and hardware.
The key thing is tool use.
The model is fixed but tool use introduces new information into the system, so once you have a model that is good enough to effectively use the full suite of tools available (debug, test, profile, etc) and write code to effectively combine them and synthesize their results for a particular task, there is nothing that it can't solve.
New models are continually ingesting new information from usage, and embedding the results from people running this process back into the base model, which makes it more efficient, gets more things right the first time, etc, but there is no step gap in functional outcomes left.
Computing and programming are a limited domain with a small fixed set of constructs that can be applied in an infinite number of combinations. Nothing ever closes the gap between the ability of the model to automate the assembly of these constructs and the necessity of expert human direction to pick what system to build out of the infinite range of possible systems.
Frontier models can keep producing very expensive one-shot demos to pitch to investors but they are never going to replace engineers.
I also switched to local qwen recently, but have to block such outgoing requests.
These 3bit ISTA quants of qwen are supposed to be 95% as accurate as the full model and runs moothly on laptops:
https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF
There are multiple independent tests of this on YouTube.
I like these quants as much as the next guy when they're all I can run but I have both R9700 with 32GB VRAM and a 5060 Ti with only 16GB VRAM and so I run both Q6 Unlsoth quants and ISTA IQ3_XXS quants and yeah there is an absolute world of difference in realworld performance.
IMHO I think it's pretty scummy behavior from the quant providers to frame these quants this way. Just be honest about where the degradation is really occurring on the non-saturated benchmarks.
Given pace of advance I'm not convinced this is a winning strategy. The models of today (open or not) are way smarter than those of a year ago.
...so we really do need open models to keep pace, not just be statically competitive at a given point in time
So when I really want to solve a problem, I just think on it. And when I need something to iterate through the things I thought of quickly, I use deepseek.
Your quantum engine aside, current open-source LLMs will not age well. They have a knowledge cutoff. If you ask them five years from now about something that didn't exist when they were trained, chances are they won't be able to provide a useful answer. They may still be useful for many applications, but their value will eventually deteriorate. Having continuous access to up-to-date open-weight LLMs is therefore about much more than simply having access to "better" models.
cecli.dev
I mainly just want to point out the value of human ingenuity coupled with a model like DS4.1 is technically enough to get most people to where they want to go, and that includes smarter open models.
and I liked it so much (and saw that the RAM drought was going to last for at least a few years) so I bought a second one to use as a general server for myself and my family.
If expectations were static you’d be fine with qwen 2.5…but nobody sane says that today.
It’s a moving target thus we need open weights to move
DeepSeek and GLM are useful, though. Even the flash variants.
Side-by-side Qwen 3.8 27B is ~6x as fast as DS4 and GLM 5.3 and faster than that if you get the speculative engine set up properly. That means you can iterate very rapidly on the same problem and for many problems this is the way to solve them. DS4 and GLM 5.3 shine when Qwen would take too many turns to solve something, they simply have more world knowledge and seem to home in on possible solutions faster in #tg but slower in wall clock time. If a particular problem is within striking range of all three then Qwen will usually be the fastest solver by far.
It also depends on how good your prompt is, whether you are setting up points for guidance or whether you expect everything to be solved in a single shot.
Wow, that sounds like a lot! Do you use/mean the regular models or flash variants here?
5257408 DeepSeek-V4-Flash-0731-UD-Q8_K_XL-00001-of-00005.gguf
49215492960 DeepSeek-V4-Flash-0731-UD-Q8_K_XL-00002-of-00005.gguf
49700372160 DeepSeek-V4-Flash-0731-UD-Q8_K_XL-00003-of-00005.gguf
49466495968 DeepSeek-V4-Flash-0731-UD-Q8_K_XL-00004-of-00005.gguf
13481997024 DeepSeek-V4-Flash-0731-UD-Q8_K_XL-00005-of-00005.gguf
49842093280 GLM-5.3-Flash-Q3_XL-3.86bpw-00001-of-00004.gguf
49644428576 GLM-5.3-Flash-Q3_XL-3.86bpw-00002-of-00004.gguf
49653319264 GLM-5.3-Flash-Q3_XL-3.86bpw-00003-of-00004.gguf
5694702080 GLM-5.3-Flash-Q3_XL-3.86bpw-00004-of-00004.gguf
20437336576 qwen3_8_27b.ninfer
Qwen benefits in two ways: all of the VRAM is on a single GPU so there is no communications other than generated tokens and new activations, that gives pretty good utilization for the card, the second is of course simply the fact that it is a much smaller model so there is less to compute. The 4090 is still power limited so it could do more but it's loud (and hot) enough as it is.
More details here:
https://jacquesmattheij.com/local-inference-with-large-langu...