  The diversity of recommendation is equally crucial as accuracy in improving
user experience. Existing studies, e.g., Determinantal Point Process (DPP) and
Maximal Marginal Relevance (MMR), employ a greedy paradigm to iteratively
select items that optimize both accuracy and diversity. However, prior methods
typically exhibit quadratic complexity, limiting their applications to the
re-ranking stage and are not applicable to other recommendation stages with a
larger pool of candidate items, such as the pre-ranking and ranking stages. In
this paper, we propose Contextual Distillation Model (CDM), an efficient
recommendation model that addresses diversification, suitable for the
deployment in all stages of industrial recommendation pipelines. Specifically,
CDM utilizes the candidate items in the same user request as context to enhance
the diversification of the results. We propose a contrastive context encoder
that employs attention mechanisms to model both positive and negative contexts.
For the training of CDM, we compare each target item with its context embedding
and utilize the knowledge distillation framework to learn the win probability
of each target item under the MMR algorithm, where the teacher is derived from
MMR outputs. During inference, ranking is performed through a linear
combination of the recommendation and student model scores, ensuring both
diversity and efficiency. We perform offline evaluations on two industrial
datasets and conduct online A/B test of CDM on the short-video platform
KuaiShou. The considerable enhancements observed in both recommendation quality
and diversity, as shown by metrics, provide strong superiority for the
effectiveness of CDM.
