If anything, you're better off supporting multiple LLMs as backup because most model providers have been so inconsistent with working all the time
You’re clearly not building a product based on an LLM.
I’m still using various old Anthropic and OpenAI models for products I’ve built and released because I can’t risk the behavior changing in unpredictable ways and the users being pissed.
It’s much easier to switch out some deterministic software than an LLM which you’ve spent a ton of time on testing and benchmarking and understanding its nuances. Changing it is like replacing an employee who’s critical to the business.
As for which model does the building... I'm not at all attached. Enough logic, and CI gates/tests live outside the whims of the LLM to be able to hotswap them any time.
In fact, most benchmarks show this! Most benchmarks have similar performance for the same classes of models.
On top of this, there are tools like open router, or even the openai SDK which trivially allows you to swap endpoints for the LLM!
If you're using the agents SDK from openai or something, then yeah it's not interchangeable but that's you doing it wrong
Because this claim is counter to my experience as well.
Of course my numbers are a sample of one and I am not spending a lot of money or time on it. Just lazily trying things on my "happen to have this" hardware. But basically trying out the Claude Code I'm used to from work but locally with a bunch of open weight models.
I can run super tiny models on my 8GB NVIDIA card. They all suck (I have to use <=~5GB models if I want "usable" ~250k context that doesn't need to use system RAM and CPU (which makes things super slow).
I've also tried a GLM 4.7-flash, which even though it's super slow (in comparison) with ~250k context and it just doesn't cut it vs. the Claude Sonnet or Opus I get to use at work. All the while these are all touted as "totally usable, Claude/ChatGPT killer!" replacements.
It's just not "there" with tool use or building software for that matter. Like, just a simple Claude "web search" fails with it. So I asked it to build itself its own "web search" functionality and it just couldn't. It made so many mistakes its just not funny any more. And it couldn't recover from them either. I retried a few times (as I didn't have python installed and it wanted to implement it using that - this happens to be new system - never mind other attempts). I spent as much time doing this (and failing) as I spent building an actual full feature at work last week w/ Sonnet.
If it can't build itself a simple web search to .md file tool/skill, how am I supposed to trust this with actual coding? I'm used to being able to point Claude at our large code base and essentially work with it like a junior doing my bidding. Maybe 5.2 is a killer game changer vs. what I was able to try out (if slowly) but you really have to show me to convince me at this point. And not with synthetic benchmarks. In those, all of the models I tried are supposedly super awesome.
Just spend $5 on OpenCode Go and give GLM 5.2 a shot if you have the time. It's not quite as good as Opus, but it's more than good enough for many tasks.
$5 the first month, then price is doubled.