  We state the problem of inverse reinforcement learning in terms of preference
elicitation, resulting in a principled (Bayesian) statistical formulation. This
generalises previous work on Bayesian inverse reinforcement learning and allows
us to obtain a posterior distribution on the agent's preferences, policy and
optionally, the obtained reward sequence, from observations. We examine the
relation of the resulting approach to other statistical methods for inverse
reinforcement learning via analysis and experimental results. We show that
preferences can be determined accurately, even if the observed agent's policy
is sub-optimal with respect to its own preferences. In that case, significantly
improved policies with respect to the agent's preferences are obtained,
compared to both other methods and to the performance of the demonstrated
policy.
