  Robots can learn to do complex tasks in simulation, but often, learned
behaviors fail to transfer well to the real world due to simulator
imperfections (the reality gap). Some existing solutions to this sim-to-real
problem, such as Grounded Action Transformation (GAT), use a small amount of
real-world experience to minimize the reality gap by grounding the simulator.
While very effective in certain scenarios, GAT is not robust on problems that
use complex function approximation techniques to model a policy. In this paper,
we introduce Reinforced Grounded Action Transformation(RGAT), a new sim-to-real
technique that uses Reinforcement Learning (RL) not only to update the target
policy in simulation, but also to perform the grounding step itself. This novel
formulation allows for end-to-end training during the grounding step, which,
compared to GAT, produces a better grounded simulator. Moreover, we show
experimentally in several MuJoCo domains that our approach leads to successful
transfer for policies modeled using neural networks.
