  A significant challenge for the practical application of reinforcement
learning in the real world is the need to specify an oracle reward function
that correctly defines a task. Inverse reinforcement learning (IRL) seeks to
avoid this challenge by instead inferring a reward function from expert
behavior. While appealing, it can be impractically expensive to collect
datasets of demonstrations that cover the variation common in the real world
(e.g. opening any type of door). Thus in practice, IRL must commonly be
performed with only a limited set of demonstrations where it can be exceedingly
difficult to unambiguously recover a reward function. In this work, we exploit
the insight that demonstrations from other tasks can be used to constrain the
set of possible reward functions by learning a "prior" that is specifically
optimized for the ability to infer expressive reward functions from limited
numbers of demonstrations. We demonstrate that our method can efficiently
recover rewards from images for novel tasks and provide intuition as to how our
approach is analogous to learning a prior.
