  Click-through rate (CTR) prediction plays an important role in personalized
recommendations. Recently, sample-level retrieval-based models (e.g., RIM) have
achieved remarkable performance by retrieving and aggregating relevant samples.
However, their inefficiency at the inference stage makes them impractical for
industrial applications. To overcome this issue, this paper proposes a
universal plug-and-play Retrieval-Oriented Knowledge (ROK) framework.
Specifically, a knowledge base, consisting of a retrieval-oriented embedding
layer and a knowledge encoder, is designed to preserve and imitate the
retrieved & aggregated representations in a decomposition-reconstruction
paradigm. Knowledge distillation and contrastive learning methods are utilized
to optimize the knowledge base, and the learned retrieval-enhanced
representations can be integrated with arbitrary CTR models in both
instance-wise and feature-wise manners. Extensive experiments on three
large-scale datasets show that ROK achieves competitive performance with the
retrieval-based CTR models while reserving superior inference efficiency and
model compatibility.
