What’s the time horizon do you think for free models matching today’s SOTA on average consumer hardware? I see people building 6k+ machines to run the best of them at the moment, which are behind SOTA by maybe 6 - 12 months or so right now.
replyOpen models lag the frontier ~3-6 months, though they're likely smaller than frontier models as well so that lag might not be fully real. Qwen 3.6 27B is very usable for average coding, and Gemma4 31b is very usable for day to day tasks.
replyThe problem there isn't the models, it's consumer hardware. Even 16GB cards aren't the norm, and even with massive improvements in per-parameter performance we probably still need 48GB memory to get models that feel smart enough to trust.
“Average” is also doing terrible things there. The “average GPU” is probably the integrated graphics on the CPU of a laptop.
replyIf you scoped it to “average gaming desktop”, double digit VRAM is pretty normal at this point. If costs came down, I imagine the higher end GPUs would start including enough VRAM for 30B-ish models.
I don't think free/open model necessarily means local. I use open code Go for $10/mo for pet projects and deepseek v4 pro is largely comparable to my workflow at work using Claude code. Obviously this wouldn't work for someone wanting to do more than just per projects (I hit my weekly quota 5 days in, on basic usage) but I'm just saying that local doesn't have to be part of the equation
reply