  Existing deep learning-based approaches for monocular 3D object detection in
autonomous driving often model the object as a rotated 3D cuboid while the
object's geometric shape has been ignored. In this work, we propose an approach
for incorporating the shape-aware 2D/3D constraints into the 3D detection
framework. Specifically, we employ the deep neural network to learn
distinguished 2D keypoints in the 2D image domain and regress their
corresponding 3D coordinates in the local 3D object coordinate first. Then the
2D/3D geometric constraints are built by these correspondences for each object
to boost the detection performance. For generating the ground truth of 2D/3D
keypoints, an automatic model-fitting approach has been proposed by fitting the
deformed 3D object model and the object mask in the 2D image. The proposed
framework has been verified on the public KITTI dataset and the experimental
results demonstrate that by using additional geometrical constraints the
detection performance has been significantly improved as compared to the
baseline method. More importantly, the proposed framework achieves
state-of-the-art performance with real time. Data and code will be available at
https://github.com/zongdai/AutoShape
