I don't think the evidence supports this claim. All the important capability gains now come from RL rollouts in real environments. You can't fake that by generating text from a teacher model. Anthropic makes a lot of noise about how China is distilling their models, but all Chinese labs can access is text outputs. They can't even get the logits. Now, compare all of the recent US open-weights models that are far behind the Chinese frontier. Why haven't they simply "distilled" all of those powerful open weight models that are available to them?

Isn't this somewhat akin to a model learning from its own inputs? Doesn't that behavior lead to model collapse? Either the model is trained on pristine data, in which case it stagnates, or it's distilled from another model, which leads to model collapse.