upvote
We don't have enough fast models, so I see this as a positive. I just test drove Gemini Flash Lite and it's crazy fast.
reply
For a coding LLM specifically, when is fast a good tradeoff for quality?
reply
> ...when is fast a good tradeoff for quality?

When it is cheaper, and the "lower quality" model is adequate for the task at hand.

Plenty of problems have a low(er) skill/intelligence floor, anyone who uses the dual-mode agent paradigm (plan, then act) figures out the second phase can be completed by a less capable model. Even when disregarding costs - speed is important here because the agent can rapidly iterate without human supervision, based on compiler errors, lint and test failures

reply
A coding agent driven by a large LLM can delegate smaller tasks to a faster model. For example searching through the codebase for references, examples, or established patterns. They are treated as tools and don't pollute the main agent's context.
reply
I wouldnt say it is, but there are circumstances when speed is helpful. I wouldn't argue that coding is one of them.
reply