For sure, myself and colleague saw this nerfing. He said - new model coming. It was incredibly obvious. I also started getting a lot of "How's Claude doing today" messages at the same time, which is presumably to get feedback on the A/B
replyI always skip those. I know it doesn’t matter in the bigger scheme of things, but I don’t want to be classified as a “good feedback provider”.
replyI’m working on a task. And the task is never to evaluate the agent performance.
And whenever you press a number, even 0, they keep your full conversation.
replySource?
replyI strongly believe it will be shit like this that will have people abandon them. They're being way too cocky for a company that pretty much didn't exist 2 years ago
replyBut the claim here is "the models are nerfed before a release", not "they A/B test quants / efforts". If it were A/B testing, it doesn't really make sense to correlate it to release windows at all.
replyThe models are nerfed [for some people] before a release
replyThey might want to see how people react to a new model depending on whether they've been nerfed.
replyIf the nerfing is lower effort (seems to be, presumably why they hide the CoT) then they may be wondering if they can get away permanently with lowering effort on the new release provided it gives a wee improvement.