Has there been any evidence of a well known provider rerouting to lower quality models?
replyLast I saw, engineers working at OpenAI denied this on HN.
I saw that someone set up a tracker that aims to record the performance of the models, and so far it has not shown any statistically significant deviation in performance for Codex, and not yet enough data for Claude: https://marginlab.ai/trackers/codex/
> Has there been any evidence of a well known provider rerouting to lower quality models?
replyThe firm [Anthropic] would deliberately degrade the model’s performance in ways that were invisible to the user.
>This is how I feel whenever I see bold all caps instructions in a system prompt or someone claims they conducted "research" and found the magic prompt template that makes the model pay out. Maybe it works some of the time but it isn't a solution that works everytime.
replyFor such thing to be useful, it's enough that they works substantially more times that not having those instructions in.
Every gambler thinks their system works, given enough chances.
reply