upvote
Adversarial distillation hasn’t been a major factor in the past year. If the SOTA stops moving, all the other labs catch up to them in 6 months to a year, and distillation into smaller models continues to close the gap between small/large models during that time
reply