undefined

points

[-]

> but everyone sees the writing on the wall

i'm... not sure? This assumes ~stagnation in task-possibility. We've had ~exponential progress for like 3+ years now; I'd have never dreamed the tooling I hammer daily would exist in my lifetime just.. 3? years ago. And it's improving daily.

Maybe Open will win, maybe Closed will keep pushing the envelope. The world here is raw enough i don't think anyone can make any significant claim other than 'holy shit this is useful and moving Fast'.

by brunooliv6 hours ago|

prev|

[-]

I hope you're right and I want you to be right, but, even seeing the current hype around local models, etc... and open-source models, I think the industry is currently under a big confusion where they see the benchmarks of things like Kimi, GLM, Qwen, they play with it via opencode, and they think like: "Wow this is pretty good, I want to deploy this". But they don't understand how the KV cache grows over time and can take almost as much memory as needed for a 30B param model, they dont understand that a quantized model WILL NOT be the same as a full precision one, and they surely don't see the engineering work needed to serve inference to even tens of customers at a decent quality and latency level.

The biggest moat of these giant labs and models is increasingly shifting towards deployment capabilities and (debatably) having better (proprietary) harnesses.

The models themselves can be impressive on benchmarks, but unless they can be served reliably to customers either at scale, hosted somewhere, or even on edge with predictable latency and memory usage, then frontier will always be leading.

by foolswisdom3 hours ago|

parent|

[-]

It sounds like you're focusing on the problems of running local models, or running models yourself, but I don't think that many people seriously expect near term improvement on that, it's definitely more just hopeful thinking there. That's not what I meant to address, and I also am in more of a "wait and see" mode.

But at this point we do expect that open weights _hosted_ options become feasible for the tasks they're using the frontier models for. And because of the lack of "legal monopoly" (intellectual property of whatever kind), they're way cheaper, not mention more flexible.

The launch of the tinker platform from Thinking Machines is an example of the "more flexibility" part that people want (and they chose to make their model open weights, maybe because this is the angle they want to push).

At this point I think it's realistic enough that the ball is in OpenAI / Anthropic's court to figure out how to respond to this threat to their business model.

That said, I think it's concerning that there are apparently only a couple of providers of hosted open weights inference, due to the complexities of doing so (per Dax from OpenCode's tweets).