  Multi-objective reinforcement learning (MORL) is an extension of ordinary,
single-objective reinforcement learning (RL) that is applicable to many
real-world tasks where multiple objectives exist without known relative costs.
We study the problem of single policy MORL, which learns an optimal policy
given the preference of objectives. Existing methods require strong assumptions
such as exact knowledge of the multi-objective Markov decision process, and are
analyzed in the limit of infinite data and time. We propose a new algorithm
called model-based envelop value iteration (EVI), which generalizes the
enveloped multi-objective $Q$-learning algorithm in Yang et al., 2019. Our
method can learn a near-optimal value function with polynomial sample
complexity and linear convergence speed. To the best of our knowledge, this is
the first finite-sample analysis of MORL algorithms.
