  We present a novel method of learning style-agnostic representation using
both style transfer and adversarial learning in the reinforcement learning
framework. The style, here, refers to task-irrelevant details such as the color
of the background in the images, where generalizing the learned policy across
environments with different styles is still a challenge. Focusing on learning
style-agnostic representations, our method trains the actor with diverse image
styles generated from an inherent adversarial style perturbation generator,
which plays a min-max game between the actor and the generator, without
demanding expert knowledge for data augmentation or additional class labels for
adversarial training. We verify that our method achieves competitive or better
performances than the state-of-the-art approaches on Procgen and Distracting
Control Suite benchmarks, and further investigate the features extracted from
our model, showing that the model better captures the invariants and is less
distracted by the shifted style. The code is available at
https://github.com/POSTECH-CVLab/style-agnostic-RL.
