Distillation attacks aren't about replacing the entire pretraining dataset with questionably sourced synthetics. It's all about post-training.
Train your own base model - but tune it off Claude output to make it perform more in line with Claude. Yoink the products of Anthropic's expensive SFT, RLHF and RLVR work for yourself by training on the outcomes.
The post-training datasets are small, but they are what controls the final model behavior.
Train a big, wide base model with a lot of potential. Mid-train or post-train that on Claude Opus 4.5 reasoning/agentic traces (i.e. Claude Code data from Chinese API resellers) to make your model approximate a high baseline of chatbot behavior, reasoning, agentic work and tool use.
Then run your own expensive SFT, RLHF and RLVR on top of that yoinked baseline to dial it in further.
Actually doing RLHF and RLVR is extremely expensive. Distillation gives you a lot of dense, high quality post-training signal for cheap. This can get your model into the basin of "the right way to tackle this kind of problem" without a frontier lab compute budget. It's a big shortcut that gets you closer to the target - you can take it from there and build on top of it with your own work.
Also, it's unclear whether "summarizing thinking tokens" actually ruins distillation, or just makes it work worse. I'd bet on the latter, really. Because it's an approximation game, and summarized reasoning is still a better approximation of true reasoning than most of what you get online and in pre-training datasets.
They are used for post-training, i.e. calibrating the model to understand and use tools/command line more effectively.
That's an increase of only a single order of magnitude, increasing my estimate of exfiltrated tokens from 0.05 to 0.15 trillion - a far cry from the 15 trillion required.
> They are used for post-training
Possibly - it may be too much data for post-training, unless further curation was done. However, this is not distillation; you know it, I know it, Dario knows it, but "Distillation Attack" is a short, memorable, sciencey-sounding, political sound-bite with enough malevolence to be deployed on the floors of congress, or by the usual fear-mongering newstainment talking heads.
Nobody is suggesting Moonshot used 15 trillion tokens of Claude data to pre-train a base model from scratch. That would be impossible and nonsensical.
This is entirely about distillation, which happens during post-training (alignment and SFT). Here, datasets are measured in millions or billions of tokens, not trillions. 50 billion Claude tokens is far, far than enough to copy Claude's reasoning logic, writing style, and tool-use ability to the pre-trained base model.
> However, this is not distillation
I don't understand how you're so caught up on the term "distillation". Distillation is using a larger model's outputs to train a (weaker) student model. Which is exactly what's happening. It's a standardized term that has been in use for a decade.
Now that Anthropic hides the real thinking tokens in a way that precludes future CoT distillation, we'll find out which side is correct based on whether Chinese AI labs close the gap or not.
My bet is they'll close the gap; nothing about frontier AI is magic, once something is shown to be possible, experienced practitioners almost always figure out how to accomplish the same feat, though not always on the same way. This is why frontier US labs keep leapfrogging each other every few months.