its not always that simple. dropping in a new model is trivial, but highly specific workflows may rely on specific _invisible_ aspects of a model. when that model gets deprecated, the workflow needs to be rebuilt/re-tuned to work with a different model.
replygoogle's inability or unwillingness to provide stable timelines for model deprecation makes it risky to build complex workflows using their models
Load-bearing (whoops) quirks were noticeable months back, but haven't most flagship models become predictable and reliable?
replyit does seem to be moving in that direction. There were really specific things (large, complex json outputs) that gemini-2.5 flash was basically the only model that seemed capable of reliably for a long period. gpt-5+ has covered the usecase for us now pretty well but still evals slightly below what 2.5 could do
reply100% agreed in the same boat right now. Feeling really screwed over by Google rn
replyYou would be surprised how much of a difference the model makes for certain niche tasks.
replyFor my use case, `gemini-3.1-flash-lite` is ~20% higher accuracy than the next best model of comparable cost (considering both proprietary and open-weight alternatives)
Well it is a bit surprising that 3.1 flash-lite could be better than deepseek-v4-pro (cheaper output and way cheaper cache so might cost less for quite a few use cases).
replyThey are not anywhere close according to pretty much every benchmark (even v4-flash is considerably ahead and its way cheaper than flash-lite). Maybe tuning prompts/tools/etc. might be useful?
Gemini flash lite family of models currently has the best ratio for price/speed/intelligence for understanding images, no real alternative AFAIK
reply"Intelligence" being what, math? Coding? Unfortunately there's a billion use cases for LLMs whose performance is not at all captured by the popular benchmarks they're all trying to maxx.
replyif you are relying on a model for a business process, it should be simple enough to benchmark on that process
reply