  Generally speaking, the model training for recommender systems can be based
on two types of data, namely explicit feedback and implicit feedback. Moreover,
because of its general availability, we see wide adoption of implicit feedback
data, such as click signal. There are mainly two challenges for the application
of implicit feedback. First, implicit data just includes positive feedback.
Therefore, we are not sure whether the non-interacted items are really negative
or positive but not displayed to the corresponding user. Moreover, the
relevance of rare items is usually underestimated since much fewer positive
feedback of rare items is collected compared with popular ones. To tackle such
difficulties, both pointwise and pairwise solutions are proposed before for
unbiased relevance learning. As pairwise learning suits well for the ranking
tasks, the previously proposed unbiased pairwise learning algorithm already
achieves state-of-the-art performance. Nonetheless, the existing unbiased
pairwise learning method suffers from high variance. To get satisfactory
performance, non-negative estimator is utilized for practical variance control
but introduces additional bias. In this work, we propose an unbiased pairwise
learning method, named UPL, with much lower variance to learn a truly unbiased
recommender model. Extensive offline experiments on real world datasets and
online A/B testing demonstrate the superior performance of our proposed method.
