  Most real-world optimization problems have multiple objectives. A system
designer needs to find a policy that trades off these objectives to reach a
desired operating point. This problem has been studied extensively in the
setting of known objective functions. We consider a more practical but
challenging setting of unknown objective functions. In industry, this problem
is mostly approached with online A/B testing, which is often costly and
inefficient. As an alternative, we propose interactive multi-objective
off-policy optimization (IMO$^3$). The key idea in our approach is to interact
with a system designer using policies evaluated in an off-policy fashion to
uncover which policy maximizes her unknown utility function. We theoretically
show that IMO$^3$ identifies a near-optimal policy with high probability,
depending on the amount of feedback from the designer and training data for
off-policy estimation. We demonstrate its effectiveness empirically on multiple
multi-objective optimization problems.
