Are you paying API rates?
replyI'm pretty confident (from internal evals and monitoring over nearly six months now; across many models) API has never been affected by A/B tests / "nerfs" / etc.
If they have different "models" depending on if you get it from Bedrock, API from Anthropic, or differing based on which subscription plan you're on, then that's fraud. Disclose the differences.
replyBut FWIW for Fable specifically, I've noticed this via API rates from Anthropic directly. Eg: added cash to my $30/month account to use Fable, and still get obvious nerfing.
That's interesting, could you share a little more about the evals please? I'm interested in knowing how sensitive they might be to nerfing given in the UI at least the incremental changes can he quite subtle
replyBut API isn't just twice as much as subscription, is it?
reply