  Imitation learning is the problem of recovering an expert policy without
access to a reward signal. Behavior cloning and GAIL are two widely used
methods for performing imitation learning. Behavior cloning converges in a few
iterations but doesn't achieve peak performance due to its inherent iid
assumption about the state-action distribution. GAIL addresses the issue by
accounting for the temporal dependencies when performing a state distribution
matching between the agent and the expert. Although GAIL is sample efficient in
the number of expert trajectories required, it is still not very sample
efficient in terms of the environment interactions needed for convergence of
the policy. Given the complementary benefits of both methods, we present a
simple and elegant method to combine both methods to enable stable and sample
efficient learning. Our algorithm is very simple to implement and integrates
with different policy gradient algorithms. We demonstrate the effectiveness of
the algorithm in low dimensional control tasks, gridworlds and in high
dimensional image-based tasks.
