Model distillation trains a smaller 'student' model to imitate a larger 'teacher', using the teacher's outputs (and sometimes internal signals) as training data
Training a smaller 'student' model to imitate a larger 'teacher', using the teacher's outputs (and sometimes internal signals) as training data, capturing much of the capability at a fraction of the size and cost.
Because it is a licensing battleground: many providers restrict using their model outputs to train competitors. Distillation is how the small-model revolution happened, compressing frontier-grade behavior into models cheap enough for high-volume and edge serving.