It certainly has helped OpenAI and Anthropic get their KV cache costs under control.
replyIt's no secret that everyone is dis-stealing from everyone else.
replyI don't see how distillation relates to using the published techniques developed by Deepseek, Moonshot, Zhipu, etc
replytouche
reply