Defining productivity is a use-case scenario, and a wildly generalized assumption for most people in this argument. Local infrastructure doesn't need to be sota for absolutely every single need for a dev lab, but it absolutely can be delivered with non-api frontier class models.
replyJust to be clear, I'm specifically talking about coding. I think local models can help with productivity today, just not coding.
replyI'm also a huge fan of local models and think it's absolutely imperative that they continue to advance so we can move off of the Anthropic/OpenAI hosted models. It's important to accurately asses where we are in that journey though.
I think the issue is generalization, if you were more specific about which local models aren’t good enough for which tasks compared to which frontier models in your experience, it’d be a lot more informative
replyI can't just go into any codebase and ask a local model to "Implement this feature: xxx" and get acceptable output. I hope to someday soon though!
replyLike the other commenter, I'm confused about the 'just not coding' conclusion. I'm using Qwen 27B on a 5090 at > 100tk/s with 150k context (which isn't enough admittedly), and DeepSeek v4 Flash with 1million context on a gb10/spark. Both of which are performing surface level, and deep needle precision infrastructure architecture. They code 24-7, stupendously.
replyIt would be interesting to hear more about how you’re actually using them. Do you have sophisticated feedback loops around the models so they can verify their work and converge on good solutions? And how do you decide what to give the 5090 vs the Spark vs a frontier model?
replyCorrectness matters much more than speed to me, but if I can get both, that’s obviously very interesting.
Local models are undeniably capable of "helping with coding" today.
replyI so want this to be true, but for the kind of coding I do (not Flask apps), it's definitely not the case. Like I said, SOTA models just barely, barely work for me. My projects are usually 100k-1M lines of Rust or Go.
replyOut of curiosity, what do you find the SOTA models are simply incapable of when it comes to your Rust and Go projects?
replyThe SOTA models now work really well in my codebases, but that's only been since Opus 4.5/4.6-ish. Prior to that, and with current local models, they simply couldn't work holistically and would just thrash around. Now I feel as if SOTA are approaching my coding levels if not surpassing it. I still need to guide on architecture, but I can see that going away within the next year or so as well.
replyThanks, that makes sense. When you said they “barely, barely worked” for you I assumed that meant something different.
replyOh yeah, that makes sense, sorry! I meant they just started working well and did not until relatively recently.
reply[dead]
reply