  End-to-end reinforcement learning on images showed significant progress in
the recent years. Data-based approach leverage data augmentation and domain
randomization while representation learning methods use auxiliary losses to
learn task-relevant features. Yet, reinforcement still struggles in visually
diverse environments full of distractions and spurious noise. In this work, we
tackle the problem of robust visual control at its core and present VIBR
(View-Invariant Bellman Residuals), a method that combines multi-view training
and invariant prediction to reduce out-of-distribution (OOD) generalization gap
for RL based visuomotor control. Our model-free approach improve baselines
performances without the need of additional representation learning objectives
and with limited additional computational cost. We show that VIBR outperforms
existing methods on complex visuo-motor control environment with high visual
perturbation. Our approach achieves state-of the-art results on the Distracting
Control Suite benchmark, a challenging benchmark still not solved by current
methods, where we evaluate the robustness to a number of visual perturbators,
as well as OOD generalization and extrapolation capabilities.
