  Our goal is to train control policies that generalize well to unseen
environments. Inspired by the Distributionally Robust Optimization (DRO)
framework, we propose DRAGEN - Distributionally Robust policy learning via
Adversarial Generation of ENvironments - for iteratively improving robustness
of policies to realistic distribution shifts by generating adversarial
environments. The key idea is to learn a generative model for environments
whose latent variables capture cost-predictive and realistic variations in
environments. We perform DRO with respect to a Wasserstein ball around the
empirical distribution of environments by generating realistic adversarial
environments via gradient ascent on the latent space. We demonstrate strong
Out-of-Distribution (OoD) generalization in simulation for (i) swinging up a
pendulum with onboard vision and (ii) grasping realistic 3D objects. Grasping
experiments on hardware demonstrate better sim2real performance compared to
domain randomization.
