I doubt this is the case. I suspect all variation in quality and output can be explained in terms of prompt differences, and changing how much thinking happens at each level, as we see in the post above.
And what about A/B testing? Anthropic doesn't even try to hide that they do this. Limited compute and A/B testing alone, would guarantee a nerfed experience some of the time. That is, you don't even need a nefarious reason.
I consider it gas lighting to deny model nerfing.
They do. I often get the message "This model is currently experiencing high demand, please try again later"
They do drop requests though? At peak times I sometimes have 1-2 requests fail with Opus; no biggie, the harness just retries it. But that, in aggregate, sheds load.
Thats not what Openai and Anthropic do though
If you want to go accuse someone of denying model nerfing, go find someone doing it and argue with them.
Edit: maybe not impossible. Side channels perhaps.
Oh, this Opus X.Y is incredible. Hmm... Now it's dumb. Rinse, repeat.
Doesn't really matter if the cause is quantization, system prompt changes, changes to thinking etc.
It's hard to see how continuously rug pulling users at every release is a good idea. Only reason I can fathom is that it's needed to keep costs inline.
Either that or it's some weird tweaking to the prompts to fix whatever. But given the prevalent sentiment that it's worse that sounds like a long shot
but realistically I think you're right, it's prompt quality and the inherent non-determinism elements of LLMs showing up; when both of those things meet you get brain dead outputs.
Everyone likes to shout "slot machines! non-determinism!" until it actually explains something, heh.
This is getting ridiculous. The number of times I have switched from one model back to the other is crazy. I do not have any proof, so its quite possible I'm just delusional. AI psychosis? Either way, it FEELS like these models work really well for a while and then fall off. Switching to the latest model on the competitor always seems better, until it too seems to taper off. By then, a new model is usually available at the competitor, so I switch.
Its possible there's no degradation at all and just a PERCEPTION of it. But to me the end user, it really doesn't matter.
People seem to underestimate how high normal variance is with these tools.