Yes they distill, but if you think you can trivially get a frontier-level model by "just" distilling from Claude's public API. you fundamentally do not understand the amount of work that goes into a modern post-training stack.
replyWithout even talking about the fact that any distillation that was done was on Opus, as the timeline of Mythos/Fable vs Kimi 3 release dates just do not match in any plausible way.
If you want to read an educated take from someone that has actually spent the last few years working on post training I recommend Nathan Lambert's: https://x.com/natolambert/status/2079616308203942332
“Distillation” is just a term of art these days. Usage of competitors’ models these days is mostly around RLHF (minus the H, I guess).
replyVery interesting, thanks!
reply> Do we need 40 people saying the same thing about how they don't feel bad and it serves them right and all that surface level stuff on every single one of these?
replyApparently we do, given that we've got govt officials wasting time, money, and effort on whining about this now.
Let's be real. In reality, it's the Chinese kid getting the 1600.
replylol, true true. Should have used a different example.
reply