  Research on debiased recommendation has shown promising results. However,
some issues still need to be handled for its application in industrial
recommendation. For example, most of the existing methods require some specific
data, architectures and training methods. In this paper, we first argue through
an online study that arbitrarily removing all the biases in industrial
recommendation may not consistently yield a desired performance improvement.
For the situation that a randomized dataset is not available, we propose a
novel self-sampling training and evaluation (SSTE) framework to achieve the
accuracy-bias tradeoff in recommendation, i.e., eliminate the harmful biases
and preserve the beneficial ones. Specifically, SSTE uses a self-sampling
module to generate some subsets with different degrees of bias from the
original training and validation data. A self-training module infers the
beneficial biases and learns better tradeoff based on these subsets, and a
self-evaluation module aims to use these subsets to construct more plausible
references to reflect the optimized model. Finally, we conduct extensive
offline experiments on two datasets to verify the effectiveness of our SSTE.
Moreover, we deploy our SSTE in homepage recommendation of a famous financial
management product called Tencent Licaitong, and find very promising results in
an online A/B test.
