In mid-2024, Anthropic made the deliberate decision to stop chasing benchmarks and focus on practical value. There was a lot of skepticism at the time, but it's proven to be a prescient decision.
replyBenchmarks are basically straight up meaningless at this point in my experience. If they mattered and were the whole story, those Chinese open models would be stomping the competition right now. Instead they're merely decent when you use them in anger for real work.
replyI'll withhold judgement until I've tried to use it.
What's your opinion of glm5 if you had a chance to use it
replyRanking Codex 5.2 ahead of plain 5.2 doesn't make sense. Codex is expressly designed for coding tasks. Not systems design, not problem analysis, and definitely not banking, but actually solving specific programming tasks (and it's very, very good at this). GPT 5.2 (non-codex) is better in every other way.
replyCodex has been post-trained for coding, including agentic coding tasks.
replyIt's certainly not impossible that the better long-horizon agentic performance in Codex overcomes any deficiencies in outright banking knowledge that Codex 5.2 has vs plain 5.2.
It could be problem specific. There are certain non program things that opus seems better than sonnet at as well
replySwapped sonnet and opus on my last reply, oops
replyMarketing team agree with benchmark score...
replyLOL come on man.
replyLet's give it a couple of days since no one believes anything from benchmarks, especially from the Gemini team (or Meta).
If we see on HN that people are willing switching their coding environment, we'll know "hot damn they cooked" otherwise this is another wiff by Google.
You can’t put Gemini and Meta in the same sentence.
Llama 4 was DOA, and Meta has given up on frontier models. Internally they’re using Claude.
replyAfter spending all that money and firing a bunch of people? Is the new group doing anything at this point?
reply