upvote
In MiMo's v2.6 public RL training run, they explicitly trained on 23 different harnesses in order to minimize overfitting on any single harness. I imagine that this technique has to be common in RL training runs given that model providers have little control over where their models are used.

https://mimo.xiaomi.com/rl/#overview

reply
It's typically easier to tune the harness around the models' existing behavior. In my experience the primary failures in models > 5B params on coding tasks are just that they aren't able to translate instructions into a correct description or implementation of the problem (will write wrong code, wrong tests, and wrong documentation). The basic tool calling and syntax of generated code is fine.
reply
I don't think this is common, though it would be interesting if ThinkingMachines/HuggingFace/OpenRouter/BaseTen offered fine-tuned variants of the most popular open models that worked with specific harnesses
reply
In hindsight I think I overestimated how common/achievable this is with the logic that if PewDiePie could do it, so could a layman https://felloai.com/pewdiepies-ai-model-beats-gpt-4o/

However, the hardware requirements are still quite enormous.

I could see a gap in the market for crowdfunded (gofundme?) post-training of open-weight models, per harness. I estimate that the one-time compute cost is probably $x000 so this scales nicely with a large number of interested developers that could reuse the weights.

reply
What does "last-mile fine-tuning on [a] harness" mean?
reply
Taking the open weight model and altering those weights in order to overfit for success on the tool shape / quirks of the harness you are running the model within.

As with most post-training this would be done via some scored/verifiable problem set (we can probably tell if the tool calls blew up during the coding trajectory) e.g. a known set of input tasks where solutions can be verified.

Last-mile aka doing in yourself (on box) vs the compute cluster that created those open weights.

https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)

I believe (at least this used to be the case) that heavily quantized models that are designed to fit on consumer hardware have inflated failure rates and this can claw back quite a bit of reliability.

reply