did they not pay them enough to get good ratings on the other 3 models?
replywhats the logic in claiming its a borked metric when everything listed is an anthropic model.
There a few benchmarks out there where all existing models have abysmal scores. So it's not actually a problem if Antrophic's older models are bad, especially if the jump to the newest model is huge, and the competition is also way below it.
replyHuh? It's a benchmark by Cognition which (1) is building their own models and (2) offers all providers and thus has an incentive to avoid hyping up any one too much.
replyBut you can just say shit now. Tokens might not be too cheap to meter but saying shit increasingly is.
reply