Have you trained models? If not then you can't really speak to how effective distillation is (answer: it is very effective)
replyWhy don't we see more American companies doing distillation? If you could make an opus 5.5 cheaply by distillation in china, you can do so even easier in America where you have more gpus.
replylol. Have you? What kind of distill you are talking about? It’s only effective in the surface, small scale model or output preference. Not useful for long reasoning chain.
replyWhy lol? Yes. All major labs use SFT... (look at job postings)
reply> Not useful for long reasoning chain.
I have a feeling you don't actually know anything about ML
deleted
reply