If you have good feedback signals, like tests/benchmarks/etc, then it is potentially better to do multiple turns where model uses that to adjust code. Which might not need as smart a model.

It depends how you use the models. These small models work great for developers who prefer to stay more in the loop, and only task the model with things that can really only be interpreted in one way.

Not to mention, they’re great for self-hosting and getting yourself to not be dependent on some API that can go down or be altered at any time.

Big models seem to mostly be good for pushing ahead the frontier - the smaller models tend to gain the frontier’s capabilities after only a handful of months anyway. Many are perfectly content remaining a few months behind the bleeding edge.