  Reinforcement learning (RL) enjoyed significant progress over the last years.
One of the most important steps forward was the wide application of neural
networks. However, architectures of these neural networks are typically
constructed manually. In this work, we study recently proposed neural
architecture search (NAS) methods for optimizing the architecture of RL agents.
We carry out experiments on the Atari benchmark and conclude that modern NAS
methods find architectures of RL agents outperforming a manually selected one.
