  Multi-stage cascade architecture exists widely in many industrial systems
such as recommender systems and online advertising, which often consists of
sequential modules including matching, pre-ranking, ranking, etc. For a long
time, it is believed pre-ranking is just a simplified version of the ranking
module, considering the larger size of the candidate set to be ranked. Thus,
efforts are made mostly on simplifying ranking model to handle the explosion of
computing power for online inference. In this paper, we rethink the challenge
of the pre-ranking system from an algorithm-system co-design view. Instead of
saving computing power with restriction of model architecture which causes loss
of model performance, here we design a new pre-ranking system by joint
optimization of both the pre-ranking model and the computing power it costs. We
name it COLD (Computing power cost-aware Online and Lightweight Deep
pre-ranking system). COLD beats SOTA in three folds: (i) an arbitrary deep
model with cross features can be applied in COLD under a constraint of
controllable computing power cost. (ii) computing power cost is explicitly
reduced by applying optimization tricks for inference acceleration. This
further brings space for COLD to apply more complex deep models to reach better
performance. (iii) COLD model works in an online learning and severing manner,
bringing it excellent ability to handle the challenge of the data distribution
shift. Meanwhile, the fully online pre-ranking system of COLD provides us with
a flexible infrastructure that supports efficient new model developing and
online A/B testing.Since 2019, COLD has been deployed in almost all products
involving the pre-ranking module in the display advertising system in Alibaba,
bringing significant improvements.
