  Although click data is widely used in search systems in practice, so far the
inherent bias, most notably position bias, has prevented it from being used in
training of a ranker for search, i.e., learning-to-rank. Recently, a number of
authors have proposed new techniques referred to as 'unbiased
learning-to-rank', which can reduce position bias and train a relatively
high-performance ranker using click data. Most of the algorithms, based on the
inverse propensity weighting (IPW) principle, first estimate the click bias at
each position, and then train an unbiased ranker with the estimated biases
using a learning-to-rank algorithm. However, there has not been a method for
pairwise learning-to-rank that can jointly conduct debiasing of click data and
training of a ranker using a pairwise loss function. In this paper, we propose
a novel algorithm, which can jointly estimate the biases at click positions and
the biases at unclick positions, and learn an unbiased ranker. Experiments on
benchmark data show that our algorithm can significantly outperform existing
algorithms. In addition, an online A/B Testing at a commercial search engine
shows that our algorithm can effectively conduct debiasing of click data and
enhance relevance ranking.
