  Robot learning methods have the potential for widespread generalization
across tasks, environments, and objects. However, these methods require large
diverse datasets that are expensive to collect in real-world robotics settings.
For robot learning to generalize, we must be able to leverage sources of data
or priors beyond the robot's own experience. In this work, we posit that
image-text generative models, which are pre-trained on large corpora of
web-scraped data, can serve as such a data source. We show that despite these
generative models being trained on largely non-robotics data, they can serve as
effective ways to impart priors into the process of robot learning in a way
that enables widespread generalization. In particular, we show how pre-trained
generative models can serve as effective tools for semantically meaningful data
augmentation. By leveraging these pre-trained models for generating appropriate
"semantic" data augmentations, we propose a system GenAug that is able to
significantly improve policy generalization. We apply GenAug to tabletop
manipulation tasks, showing the ability to re-target behavior to novel
scenarios, while only requiring marginal amounts of real-world data. We
demonstrate the efficacy of this system on a number of object manipulation
problems in the real world, showing a 40% improvement in generalization to
novel scenes and objects.
