  Simulation based learning often provides a cost-efficient recourse to
reinforcement learning applications in robotics. However, simulators are
generally incapable of accurately replicating real-world dynamics, and thus
bridging the sim2real gap is an important problem in simulation based learning.
Current solutions to bridge the sim2real gap involve hybrid simulators that are
augmented with neural residual models. Unfortunately, they require a separate
residual model for each individual environment configuration (i.e., a fixed
setting of environment variables such as mass, friction etc.), and thus are not
transferable to new environments quickly. To address this issue, we propose a
Reverse Action Transformation (RAT) policy which learns to imitate simulated
policies in the real-world. Once learnt from a single environment, RAT can then
be deployed on top of a Universal Policy Network to achieve zero-shot
adaptation to new environments. We empirically evaluate our approach in a set
of continuous control tasks and observe its advantage as a few-shot and
zero-shot learner over competing baselines.
