  Learning from visual observations is a fundamental yet challenging problem in
Reinforcement Learning (RL). Although algorithmic advances combined with
convolutional neural networks have proved to be a recipe for success, current
methods are still lacking on two fronts: (a) data-efficiency of learning and
(b) generalization to new environments. To this end, we present Reinforcement
Learning with Augmented Data (RAD), a simple plug-and-play module that can
enhance most RL algorithms. We perform the first extensive study of general
data augmentations for RL on both pixel-based and state-based inputs, and
introduce two new data augmentations - random translate and random amplitude
scale. We show that augmentations such as random translate, crop, color jitter,
patch cutout, random convolutions, and amplitude scale can enable simple RL
algorithms to outperform complex state-of-the-art methods across common
benchmarks. RAD sets a new state-of-the-art in terms of data-efficiency and
final performance on the DeepMind Control Suite benchmark for pixel-based
control as well as OpenAI Gym benchmark for state-based control. We further
demonstrate that RAD significantly improves test-time generalization over
existing methods on several OpenAI ProcGen benchmarks. Our RAD module and
training code are available at https://www.github.com/MishaLaskin/rad.
