The trick is to rarely switch, or switch at task boundaries. Often the conclusion of routing is actually "this one model is actually at the pareto front for this task, just use it always".
replyBut then it's better to just not have a gateway switch models at all.
replyJust have the harness able to choose which model its sub-agents use, then tell it how to split up tasks and which models to use when doing so.
That is another way to do. Or we can automatically figure out which models the subagents should be using for you. And update them as new models come out and the work your subagents do changes. More than one way to skin a cat.
replyand caching is related to performance too ofc
reply