  Visual reinforcement learning (RL), which makes decisions directly from
high-dimensional visual inputs, has demonstrated significant potential in
various domains. However, deploying visual RL techniques in the real world
remains challenging due to their low sample efficiency and large generalization
gaps. To tackle these obstacles, data augmentation (DA) has become a widely
used technique in visual RL for acquiring sample-efficient and generalizable
policies by diversifying the training data. This survey aims to provide a
timely and essential review of DA techniques in visual RL in recognition of the
thriving development in this field. In particular, we propose a unified
framework for analyzing visual RL and understanding the role of DA in it. We
then present a principled taxonomy of the existing augmentation techniques used
in visual RL and conduct an in-depth discussion on how to better leverage
augmented data in different scenarios. Moreover, we report a systematic
empirical evaluation of DA-based techniques in visual RL and conclude by
highlighting the directions for future research. As the first comprehensive
survey of DA in visual RL, this work is expected to offer valuable guidance to
this emerging field.
