  We present an end-to-end Reinforcement Learning(RL) framework for robotic
manipulation tasks, using a robust and efficient keypoints representation. The
proposed method learns keypoints from camera images as the state
representation, through a self-supervised autoencoder architecture. The
keypoints encode the geometric information, as well as the relationship of the
tool and target in a compact representation to ensure efficient and robust
learning. After keypoints learning, the RL step then learns the robot motion
from the extracted keypoints state representation. The keypoints and RL
learning processes are entirely done in the simulated environment. We
demonstrate the effectiveness of the proposed method on robotic manipulation
tasks including grasping and pushing, in different scenarios. We also
investigate the generalization capability of the trained model. In addition to
the robust keypoints representation, we further apply domain randomization and
adversarial training examples to achieve zero-shot sim-to-real transfer in
real-world robotic manipulation tasks.
