  Explicit engineering of reward functions for given environments has been a
major hindrance to reinforcement learning methods. While Inverse Reinforcement
Learning (IRL) is a solution to recover reward functions from demonstrations
only, these learned rewards are generally heavily \textit{entangled} with the
dynamics of the environment and therefore not portable or \emph{robust} to
changing environments. Modern adversarial methods have yielded some success in
reducing reward entanglement in the IRL setting. In this work, we leverage one
such method, Adversarial Inverse Reinforcement Learning (AIRL), to propose an
algorithm that learns hierarchical disentangled rewards with a policy over
options. We show that this method has the ability to learn \emph{generalizable}
policies and reward functions in complex transfer learning tasks, while
yielding results in continuous control benchmarks that are comparable to those
of the state-of-the-art methods.
