  As Learning-to-Rank (LTR) approaches primarily seek to improve ranking
quality, their output scores are not scale-calibrated by design. This
fundamentally limits LTR usage in score-sensitive applications. Though a simple
multi-objective approach that combines a regression and a ranking objective can
effectively learn scale-calibrated scores, we argue that the two objectives are
not necessarily compatible, which makes the trade-off less ideal for either of
them. In this paper, we propose a practical regression compatible ranking (RCR)
approach that achieves a better trade-off, where the two ranking and regression
components are proved to be mutually aligned. Although the same idea applies to
ranking with both binary and graded relevance, we mainly focus on binary labels
in this paper. We evaluate the proposed approach on several public LTR
benchmarks and show that it consistently achieves either best or competitive
result in terms of both regression and ranking metrics, and significantly
improves the Pareto frontiers in the context of multi-objective optimization.
Furthermore, we evaluated the proposed approach on YouTube Search and found
that it not only improved the ranking quality of the production pCTR model, but
also brought gains to the click prediction accuracy. The proposed approach has
been successfully deployed in the YouTube production system.
