  We identify two issues with the family of algorithms based on the Adversarial
Imitation Learning framework. The first problem is implicit bias present in the
reward functions used in these algorithms. While these biases might work well
for some environments, they can also lead to sub-optimal behavior in others.
Secondly, even though these algorithms can learn from few expert
demonstrations, they require a prohibitively large number of interactions with
the environment in order to imitate the expert for many real-world
applications. In order to address these issues, we propose a new algorithm
called Discriminator-Actor-Critic that uses off-policy Reinforcement Learning
to reduce policy-environment interaction sample complexity by an average factor
of 10. Furthermore, since our reward function is designed to be unbiased, we
can apply our algorithm to many problems without making any task-specific
adjustments.
