  Policies for complex visual tasks have been successfully learned with deep
reinforcement learning, using an approach called deep Q-networks (DQN), but
relatively large (task-specific) networks and extensive training are needed to
achieve good performance. In this work, we present a novel method called policy
distillation that can be used to extract the policy of a reinforcement learning
agent and train a new network that performs at the expert level while being
dramatically smaller and more efficient. Furthermore, the same method can be
used to consolidate multiple task-specific policies into a single policy. We
demonstrate these claims using the Atari domain and show that the multi-task
distilled agent outperforms the single-task teachers as well as a
jointly-trained DQN agent.
