  Information Retrieval (IR) and Recommender Systems (RS) tasks are moving from
computing a ranking of final results based on a single metric to
multi-objective problems. Solving these problems leads to a set of
Pareto-optimal solutions, known as Pareto frontier, in which no objective can
be further improved without hurting the others. In principle, all the points on
the Pareto frontier are potential candidates to represent the best model
selected with respect to the combination of two, or more, metrics. To our
knowledge, there are no well-recognized strategies to decide which point should
be selected on the frontier. In this paper, we propose a novel, post-hoc,
theoretically-justified technique, named "Population Distance from Utopia"
(PDU), to identify and select the one-best Pareto-optimal solution from the
frontier. In detail, PDU analyzes the distribution of the points by
investigating how far each point is from its utopia point (the ideal
performance for the objectives). The possibility of considering fine-grained
utopia points allows PDU to select solutions tailored to individual user
preferences, a novel feature we call "calibration". We compare PDU against
existing state-of-the-art strategies through extensive experiments on tasks
from both IR and RS. Experimental results show that PDU and combined with
calibration notably impact the solution selection. Furthermore, the results
show that the proposed framework selects a solution in a principled way,
irrespective of its position on the frontier, thus overcoming the limits of
other strategies.
