  In this article we introduce the Arcade Learning Environment (ALE): both a
challenge problem and a platform and methodology for evaluating the development
of general, domain-independent AI technology. ALE provides an interface to
hundreds of Atari 2600 game environments, each one different, interesting, and
designed to be a challenge for human players. ALE presents significant research
challenges for reinforcement learning, model learning, model-based planning,
imitation learning, transfer learning, and intrinsic motivation. Most
importantly, it provides a rigorous testbed for evaluating and comparing
approaches to these problems. We illustrate the promise of ALE by developing
and benchmarking domain-independent agents designed using well-established AI
techniques for both reinforcement learning and planning. In doing so, we also
propose an evaluation methodology made possible by ALE, reporting empirical
results on over 55 different games. All of the software, including the
benchmark agents, is publicly available.
