upvote
What gap?
reply
The gap is so big you can't see it.
reply
Yeah, this model is ~comparable to Opus 4.6. That was, not too long ago, a frontier model.

I didn't find any non-sloppy factual report, but this channel has a good video on it: https://www.youtube.com/watch?v=OdMpNiByntM

reply
Exactly. It seems to not really be a permanent gap but more a time gap. Locally you're about a year or so behind SOTA. Is that terrible? Not really, a year ago cloud model users were raving about what they had.

There's some things like knowledge breadth that really need a huge model but those things can be compensated for with web lookups (which is preferable for source validation and currency anyway)

reply
Yeah, a good balance would be a local reasoning model paired w/ a knowledge pack + web-based searches. You want your obscure coding language? Provide a pre-vetted knowledge pack locally that it knows how to use. Same with any other domain-specific knowledge.
reply
Yes especially now that local context isn't so costly anymore. With llama3 it would kill my vram just to have a few thousand tokens. With recent models it's hardly an impact anymore. Not sure how they manage that but it's great. It still takes more processing but I also run an older card that is not very good at prefill.
reply
> Locally you're about a year or so behind SOTA

If that holds, I'll be able to do 99% of my work with local models around March 2027.

reply
Would local models catch up if frontier models paused new training for a year?

I'm not sure for. It might just be you can get these nice local models because of distillation from a much more powerful model.

reply
Adversarial distillation hasn’t been a major factor in the past year. If the SOTA stops moving, all the other labs catch up to them in 6 months to a year, and distillation into smaller models continues to close the gap between small/large models during that time
reply
yep came here to say this
reply
Weirdly, I use lab models to make the local models useful. Nothing like getting Claude to spin up the system that will put it out of work.
reply
Yes me too lol. Most of my Claude work is about fine-tuning my local setup. Especially now as I'm rearranging it all for strata.

My goal is not to be 100% local but to have all my personal data on local models only. Claude I just use to build stuff so it doesn't contain much data about me.

reply
To be fair, the models putting us out of work were trained on our own output as well.
reply