Debugging and diagnosis is very tool call heavy, whether that's grepping / transforming logs, calling out to profilers/tracers, or even just writing up incident reports.
replyBug diagnostics is about being okay at coding but better at tooling.
Given a good diagnostic report, it can be handed to opus for the fix.
Opus is okay at writing reports, but it still regularly gets table widths wrong in typst documents, leaving the last column full of text but only a handful of characters wide.
Gemini 3.5 flash is better than fable at tool calling. Tool calling is probably one of the easier things to do post training for.
replyI wonder if we'll start to see that pattern with every new release. Tool use likely changes rapidly, so the newest, rather than most intelligent, model may always have an edge.
replyWhat you mean.. The tools are all just invoking bash and terminal/cli cmds and http requests. Paradigms that have existed and stayed mostly unchanged for decades.
replyThese do make up a huge % of tool calls, but I don't think these make up a huge % of tool call failures.
replyI see models fail on tool calls that involve API requests to a specific API, internal or cloned Makefile calls, npm run commands, etc.
This sounds... kind of useless? Really good JSON or similar constrained decoder performance is interesting, but normal decoder > tool validator loop with good error message > tool retry is almost always able to get a tool to work second try, and input is cached so it's not expensive.
replyThings are not always that simple, eg https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/
replyThe avg coding session has hundreds or thousands of tool calls. Even a 5% failure rate noticeably notches up token use and cost. See Gemini.
replyYes, but each tool call has a different failure %. The tool calls that make up the majority of volume like grep are going to have nowhere near a 5% failure. A custom user-defined skill having a 5% failure rate is probably fine.
reply