  Atari games have been a long-standing benchmark in the reinforcement learning
(RL) community for the past decade. This benchmark was proposed to test general
competency of RL algorithms. Previous work has achieved good average
performance by doing outstandingly well on many games of the set, but very
poorly in several of the most challenging games. We propose Agent57, the first
deep RL agent that outperforms the standard human benchmark on all 57 Atari
games. To achieve this result, we train a neural network which parameterizes a
family of policies ranging from very exploratory to purely exploitative. We
propose an adaptive mechanism to choose which policy to prioritize throughout
the training process. Additionally, we utilize a novel parameterization of the
architecture that allows for more consistent and stable learning.
