  Over the past few years, the acceleration of computing resources and research
in deep learning has led to significant practical successes in a range of
tasks, including in particular in computer vision. Building on these advances,
reinforcement learning has also seen a leap forward with the emergence of
agents capable of making decisions directly from visual observations. Despite
these successes, the over-parametrization of neural architectures leads to
memorization of the data used during training and thus to a lack of
generalization. Reinforcement learning agents based on visual inputs also
suffer from this phenomenon by erroneously correlating rewards with unrelated
visual features such as background elements. To alleviate this problem, we
introduce a new regularization technique consisting of channel-consistent local
permutations (CLOP) of the feature maps. The proposed permutations induce
robustness to spatial correlations and help prevent overfitting behaviors in
RL. We demonstrate, on the OpenAI Procgen Benchmark, that RL agents trained
with the CLOP method exhibit robustness to visual changes and better
generalization properties than agents trained using other state-of-the-art
regularization techniques. We also demonstrate the effectiveness of CLOP as a
general regularization technique in supervised learning.
