  One fundamental difficulty in robotic learning is the sim-real gap problem.
In this work, we propose to use segmentation as the interface between
perception and control, as a domain-invariant state representation. We identify
two sources of sim-real gap, one is dynamics sim-real gap, the other is visual
sim-real gap. To close dynamics sim-real gap, we propose to use closed-loop
control. For complex task with segmentation mask input, we further propose to
learn a closed-loop model-free control policy with deep neural network using
imitation learning. To close visual sim-real gap, we propose to learn a
perception model in real environment using simulated target plus real
background image, without using any real world supervision.
  We demonstrate this methodology in eye-in-hand grasping task. We train a
closed-loop control policy model that taking the segmentation as input using
simulation. We show that this control policy is able to transfer from
simulation to real environment. The closed-loop control policy is not only
robust with respect to discrepancies between the dynamic model of the simulated
and real robot, but also is able to generalize to unseen scenarios where the
target is moving and even learns to recover from failures. We train the
perception segmentation model using training data generated by composing real
background images with simulated images of the target. Combining the control
policy learned from simulation with the perception model, we achieve an
impressive $\bf{88\%}$ success rate in grasping a tiny sphere with a real
robot.
