  Detecting and localizing objects in the real 3D space, which plays a crucial
role in scene understanding, is particularly challenging given only a single
RGB image due to the geometric information loss during imagery projection. We
propose MonoGRNet for the amodal 3D object detection from a monocular RGB image
via geometric reasoning in both the observed 2D projection and the unobserved
depth dimension. MonoGRNet is a single, unified network composed of four
task-specific subnetworks, responsible for 2D object detection, instance depth
estimation (IDE), 3D localization and local corner regression. Unlike the
pixel-level depth estimation that needs per-pixel annotations, we propose a
novel IDE method that directly predicts the depth of the targeting 3D bounding
box's center using sparse supervision. The 3D localization is further achieved
by estimating the position in the horizontal and vertical dimensions. Finally,
MonoGRNet is jointly learned by optimizing the locations and poses of the 3D
bounding boxes in the global context. We demonstrate that MonoGRNet achieves
state-of-the-art performance on challenging datasets.
