Why does your benchmark have Opus 5 scoring the highest?
replyBecause Opus 5 is the best at their benchmark.
replyyeah, it performed best for whatever reason. More specifically, it managed to deduce the meaning of the hack statement in the language better than Opus 5.5 did.
replyHow many runs you make for each model?
replyDo you report an average score or the highest.
2 runs for each task, for each task it's either a pass/fail (the small green/red dots show each task pass/fail)
reply