- Corrupted/dropped Gemini thought_signature (reasoning_details) in tool loops
- Blinded model.api.id = "contestant" bypasses gemini.txt and serves Claude's prompt (default.txt)
- Anthropic-only prompt caching
Prompt was:
> I read on hacker news the following, which does not match my expectations. Check if something in the harness or bench is hurting gemini perf.
followed by your post.
> anthropic only prompt caching
Anthropic has custom prompt cache requirements, but we've verified all providers are working well.
> Blinded model.api.id = "contestant" bypasses gemini.txt and serves Claude's prompt (default.txt)
This is wrong. We're serving default.txt which is opencode's default prompt. We are not serving a custom prompt for any model, including Gemini.
> Corrupted/dropped Gemini thought_signature (reasoning_details) in tool loops
We blind the reasoning details in the frontend because this would leak information. Some models summarize thinking while others provide raw thinking. I'll check if we're corrupting the thought signature that's actually sent to the backend.
So far this sort of looks like an example of Gemini being bad. Was this Argon?
Their agents leaderboard is literally just randomizing the agent and looking at metadata like tool call errors. In my opinion, it's poor quality data in high volume.
Their web dev leaderboard is in cloud containers, most people don't run agents like that.
I ran a statistical test on our data. Deepseek vs. Argon. Posterior probability DeepSeek 4.1 Flash is stronger: 99.1%.
It's possible, but we are getting real prompts from people on real projects, and the instructions about the battle are pretty minimal.
Argon in low and medium mode had higher quality results than it did when running in high mode, almost every time. My tasks involved the agent finding some details in a raster image and then mark it up with predictions about what it found.
Argon on high would do the work, and then a whole bunch of other stuff i did not ask for. All that extra broke everything else downstream from the analysis.
However, doing the same work with Argon in low effort was on par with results from Haiku 5.5 (xhigh) and GPT-6.1-Terra (medium). So, Argon low works better than Argon high, but for my tasks -- a very weak model relative to how google positions it.
I'm ready for Geimini Unobtanium!
what does "local" coding agents mean? Surely you're not hosting it..
But yes, Opus is a tier above Wonder and Argon with very high confidence.