  There are a lot of real-world black-box optimization problems that need to
optimize multiple criteria simultaneously. However, in a multi-objective
optimization (MOO) problem, identifying the whole Pareto front requires the
prohibitive search cost, while in many practical scenarios, the decision maker
(DM) only needs a specific solution among the set of the Pareto optimal
solutions. We propose a Bayesian optimization (BO) approach to identifying the
most preferred solution in the MOO with expensive objective functions, in which
a Bayesian preference model of the DM is adaptively estimated by an interactive
manner based on the two types of supervisions called the pairwise preference
and improvement request. To explore the most preferred solution, we define an
acquisition function in which the uncertainty both in the objective functions
and the DM preference is incorporated. Further, to minimize the interaction
cost with the DM, we also propose an active learning strategy for the
preference estimation. We empirically demonstrate the effectiveness of our
proposed method through the benchmark function optimization and the
hyper-parameter optimization problems for machine learning models.
