  A key challenge with machine learning approaches for ranking is the gap
between the performance metrics of interest and the surrogate loss functions
that can be optimized with gradient-based methods. This gap arises because
ranking metrics typically involve a sorting operation which is not
differentiable w.r.t. the model parameters. Prior works have proposed
surrogates that are loosely related to ranking metrics or simple smoothed
versions thereof, and often fail to scale to real-world applications. We
propose PiRank, a new class of differentiable surrogates for ranking, which
employ a continuous, temperature-controlled relaxation to the sorting operator
based on NeuralSort [1]. We show that PiRank exactly recovers the desired
metrics in the limit of zero temperature and further propose a divide
and-conquer extension that scales favorably to large list sizes, both in theory
and practice. Empirically, we demonstrate the role of larger list sizes during
training and show that PiRank significantly improves over comparable approaches
on publicly available internet-scale learning-to-rank benchmarks.
