  We study the problem of smooth imitation learning for online sequence
prediction, where the goal is to train a policy that can smoothly imitate
demonstrated behavior in a dynamic and continuous environment in response to
online, sequential context input. Since the mapping from context to behavior is
often complex, we take a learning reduction approach to reduce smooth imitation
learning to a regression problem using complex function classes that are
regularized to ensure smoothness. We present a learning meta-algorithm that
achieves fast and stable convergence to a good policy. Our approach enjoys
several attractive properties, including being fully deterministic, employing
an adaptive learning rate that can provably yield larger policy improvements
compared to previous approaches, and the ability to ensure stable convergence.
Our empirical results demonstrate significant performance gains over previous
approaches.
