  Generative adversarial imitation learning (GAIL) demonstrates tremendous
success in practice, especially when combined with neural networks. Different
from reinforcement learning, GAIL learns both policy and reward function from
expert (human) demonstration. Despite its empirical success, it remains unclear
whether GAIL with neural networks converges to the globally optimal solution.
The major difficulty comes from the nonconvex-nonconcave minimax optimization
structure. To bridge the gap between practice and theory, we analyze a
gradient-based algorithm with alternating updates and establish its sublinear
convergence to the globally optimal solution. To the best of our knowledge, our
analysis establishes the global optimality and convergence rate of GAIL with
neural networks for the first time.
