I second that. Gemini 3.5 Flash rocks the benchmark charts but is terrible as an agent. Horrible instruction adherence and makes WAY too many tool calls
replywhich cheap models have you found work best as agents?
replyMost of the bigger open weight models are pretty good. You can get them per-token from companies like Fireworks or OpenCode
replyThen why are they publishing the benchmarks which makes them look worse than GLM 5.2?
replyBecause it's still informative
replyI'm not sure why I'm being downvoted but I didn't mean it in a negative way.
replyFor such announcement, I would expect them to give me clues on when I should use this model and in which cases it's the best one.
The benchmarks that they share doesn't indicate that it's cheaper to run than other models, or can fit in my local machine, or excels in a specific vertical.
After reading the comments here and X, I can see it being the top-3 multi-modal open-source model though.
being close is still impressive, especially for their first (released) model
replygives me hope that the training moat is even smaller than we thought