  Imitation learning enables agents to reuse and adapt the hard-won expertise
of others, offering a solution to several key challenges in learning behavior.
Although it is easy to observe behavior in the real-world, the underlying
actions may not be accessible. We present a new method for imitation solely
from observations that achieves comparable performance to experts on
challenging continuous control tasks while also exhibiting robustness in the
presence of observations unrelated to the task. Our method, which we call FORM
(for "Future Observation Reward Model") is derived from an inverse RL objective
and imitates using a model of expert behavior learned by generative modelling
of the expert's observations, without needing ground truth actions. We show
that FORM performs comparably to a strong baseline IRL method (GAIL) on the
DeepMind Control Suite benchmark, while outperforming GAIL in the presence of
task-irrelevant features.
