  Agents that can learn to imitate given video observation -- \emph{without
direct access to state or action information} are more applicable to learning
in the natural world. However, formulating a reinforcement learning (RL) agent
that facilitates this goal remains a significant challenge. We approach this
challenge using contrastive training to learn a reward function comparing an
agent's behaviour with a single demonstration. We use a Siamese recurrent
neural network architecture to learn rewards in space and time between motion
clips while training an RL policy to minimize this distance. Through
experimentation, we also find that the inclusion of multi-task data and
additional image encoding losses improve the temporal consistency of the
learned rewards and, as a result, significantly improves policy learning. We
demonstrate our approach on simulated humanoid, dog, and raptor agents in 2D
and a quadruped and a humanoid in 3D. We show that our method outperforms
current state-of-the-art techniques in these environments and can learn to
imitate from a single video demonstration.
