  6-DoF robotic grasping is a long-lasting but unsolved problem. Recent methods
utilize strong 3D networks to extract geometric grasping representations from
depth sensors, demonstrating superior accuracy on common objects but perform
unsatisfactorily on photometrically challenging objects, e.g., objects in
transparent or reflective materials. The bottleneck lies in that the surface of
these objects can not reflect back accurate depth due to the absorption or
refraction of light. In this paper, in contrast to exploiting the inaccurate
depth data, we propose the first RGB-only 6-DoF grasping pipeline called
MonoGraspNet that utilizes stable 2D features to simultaneously handle
arbitrary object grasping and overcome the problems induced by photometrically
challenging objects. MonoGraspNet leverages keypoint heatmap and normal map to
recover the 6-DoF grasping poses represented by our novel representation
parameterized with 2D keypoints with corresponding depth, grasping direction,
grasping width, and angle. Extensive experiments in real scenes demonstrate
that our method can achieve competitive results in grasping common objects and
surpass the depth-based competitor by a large margin in grasping
photometrically challenging objects. To further stimulate robotic manipulation
research, we additionally annotate and open-source a multi-view and multi-scene
real-world grasping dataset, containing 120 objects of mixed photometric
complexity with 20M accurate grasping labels.
