I’m an LLM fan, but from an engineering perspective the idea of building atop services that palpably fluctuate in capacity, performance, and capability is nutty.
replyEven with minor automation I feel like I can watch OpenAI and Anthropic engineers fiddling in real-time. Tuesdays behaviour changes by Thursday, 10AMs production isn’t possible at 11:30AM. Nutty.
I chilled significantly on using Google for anything to do with business due to API (and offering) stability. (Still use Google for personal things.) But AI models seem orders of magnitude more fluid, so to my risk-averse eye, they're nothing I'd base my own business on.
replyImagine having a business where you're at the mercy of the fluctuations in capacity, performance, and capability that your human employees display!
replySince I started running my own inference server, I've had zero degradation that I didn't do myself. Basically the only time I see it get worse is if I drop one of the quants.
replyWhich is what I suspect the providers are doing to fit more inference on the same amount of hardware over time.
Interesting, Claude might be doing better since I last checked:
replyhttps://marginlab.ai/trackers/claude-code-historical-perform...
There were at least a couple of these degradation trackers.