  Deep reinforcement learning (RL) has enabled training action-selection
policies, end-to-end, by learning a function which maps image pixels to action
outputs. However, it's application to visuomotor robotic policy training has
been limited because of the challenge of large-scale data collection when
working with physical hardware. A suitable visuomotor policy should perform
well not just for the task-setup it has been trained for, but also for all
varieties of the task, including novel objects at different viewpoints
surrounded by task-irrelevant objects. However, it is impractical for a robotic
setup to sufficiently collect interactive samples in a RL framework to
generalize well to novel aspects of a task. In this work, we demonstrate that
by using adversarial training for domain transfer, it is possible to train
visuomotor policies based on RL frameworks, and then transfer the acquired
policy to other novel task domains. We propose to leverage the deep RL
capabilities to learn complex visuomotor skills for uncomplicated task setups,
and then exploit transfer learning to generalize to new task domains provided
only still images of the task in the target domain. We evaluate our method on
two real robotic tasks, picking and pouring, and compare it to a number of
prior works, demonstrating its superiority.
