Distillation is a superficial step and doesn't need a lot of data, it's not "stealing the model" like they want everyone to believe. 99% of work is already done by that point. That said, it's pretty clear K3 has Claude's data in the training set (either Opus or Fable), as it repeats Anthropic's prompt injections. (not that it matters to anyone besides Anthropic themselves)
replyIf by "distill" they mean "used it for fine-tuning" then they might have used it in the final stages of fine-tuning of Kimi K3. I image they might have already been using Opus, and when Fable became available it was easy to switch over to it
replyIt would have been a tiny part of the overall training, given the timeline
A month seems plenty long enough. They're not rebuilding the entire model from scratch. It's just getting Fable to act as a teacher model for some of the final reinforcement learning on the base that Kimi already had.
replyI think that would also be a bad idea, as all models opus 4.6 got increasingly smarter, but also crappier at following instructions or genuinely assisting.
replyThey just try to figure out what the goal is and hyper focus on solving it.
Heh, even just telling fable don't commit doesn't work half the times, let alone more complex instructions.
Create a couple thousand Claude max accounts and split the work amongst them perhaps.
replyThat's crazy income for Anthropic to invest into Legendos.
replyHonestly, it wouldn't surprise me if they just found evidence of distillation once in 2025 against some Chinese AI lab, and they've been lying about the rest to create a narrative.
replyI also heard that K3 stole the 2020 election, among other things.
reply