  Learning to Rank (LTR) technique is ubiquitous in the Information Retrieval
system nowadays, especially in the Search Ranking application. The query-item
relevance labels typically used to train the ranking model are often noisy
measurements of human behavior, e.g., product rating for product search. The
coarse measurements make the ground truth ranking non-unique with respect to a
single relevance criterion. To resolve ambiguity, it is desirable to train a
model using many relevance criteria, giving rise to Multi-Label LTR (MLLTR).
Moreover, it formulates multiple goals that may be conflicting yet important to
optimize for simultaneously, e.g., in product search, a ranking model can be
trained based on product quality and purchase likelihood to increase revenue.
In this research, we leverage the Multi-Objective Optimization (MOO) aspect of
the MLLTR problem and employ recently developed MOO algorithms to solve it.
Specifically, we propose a general framework where the information from labels
can be combined in a variety of ways to meaningfully characterize the trade-off
among the goals. Our framework allows for any gradient based MOO algorithm to
be used for solving the MLLTR problem. We test the proposed framework on two
publicly available LTR datasets and one e-commerce dataset to show its
efficacy.
