  We propose a new 6-DoF grasp pose synthesis approach from 2D/2.5D input based
on keypoints. Keypoint-based grasp detector from image input has demonstrated
promising results in the previous study, where the additional visual
information provided by color images compensates for the noisy depth
perception. However, it relies heavily on accurately predicting the location of
keypoints in the image space. In this paper, we devise a new grasp generation
network that reduces the dependency on precise keypoint estimation. Given an
RGB-D input, our network estimates both the grasp pose from keypoint detection
as well as scale towards the camera. We further re-design the keypoint output
space in order to mitigate the negative impact of keypoint prediction noise to
Perspective-n-Point (PnP) algorithm. Experiments show that the proposed method
outperforms the baseline by a large margin, validating the efficacy of our
approach. Finally, despite trained on simple synthetic objects, our method
demonstrate sim-to-real capacity by showing competitive results in real-world
robot experiments.
