upvote

points

by jsnell1 days ago |
comments
upvoteby breakwaterlabs1 days ago|
next
[-]
> Because on that specific benchmark (FrontierCode 2.0), Medium is Carbon's best effort level

I am referring to the chart that cross-compares multiple benchmarks, which Artificial Analysis clearly shows Carbon-high as superior on. Further, AA's FrontierCode chart places Carbon-high as having a higher score on Frontier 2.0 than Carbon-med (https://artificialanalysis.ai/evaluations/frontiercode-2-0?m...)

Where it gets very interesting is when you look at the Cognition "FrontierCode" publish date (2026.10.27), and the revision as of 10.28 in web archive (https://web.archive.org/web/20261028175237/https://cognition...). This revision has a VERY different Carbon effort / intelligence curve where high beats medium.

It is only with the latest revision, as of today, that all of a sudden Carbon's benchmark curve goes cone-shaped. Curious, indeed....

I'd encourage you to take a look at a third-party's take on this and see whether Anthropic's self-serving report looks "on the level" after doing so: https://artificialanalysis.ai/?models=claude-fable-5-2-high%...

reply
upvoteby phoghed1 days ago|
prev|
[-]
> Do you expect that OpenAI would grant such access in this case?I

Do you expect that nobody at Anthropic has such access? Trillion dollar company can’t manage this basic level of opposition research?

reply
upvoteby judge20201 days ago|
parent|
next
[-]
Best not to give special access to your biggest direct competitor beyond accessing your product at typical API prices.
reply
upvoteby catlifeonmars1 days ago|
parent|
[-]
Probably not for lack of trying though. The incentives for corporate espionage are strong.
reply
upvoteby Tadpole91811 days ago|
parent|
prev|
[-]
Isn't it against TOS for OpenAI to use their models in the development of competing models? So, any employee who does so would be opening themselves up to civil liability at AI scales. IANAL, obviously, so genuinely curious.
reply
upvoteby kragen1 days ago|
parent|
[-]
Maybe they contract out the work to other companies in other countries who promise not to access OpenAI and then sign up to do things they couldn't do without accessing OpenAI? Like independent promoter payola.
reply