  3D object detection with a single image is an essential and challenging task
for autonomous driving. Recently, keypoint-based monocular 3D object detection
has made tremendous progress and achieved great speed-accuracy trade-off.
However, there still exists a huge gap with LIDAR-based methods in terms of
accuracy. To improve their performance without sacrificing efficiency, we
propose a sort of lightweight feature pyramid network called Lite-FPN to
achieve multi-scale feature fusion in an effective and efficient way, which can
boost the multi-scale detection capability of keypoint-based detectors.
Besides, the misalignment between classification score and localization
precision is further relieved by introducing a novel regression loss named
attention loss. With the proposed loss, predictions with high confidence but
poor localization are treated with more attention during the training phase.
Comparative experiments based on several state-of-the-art keypoint-based
detectors on the KITTI dataset show that our proposed methods manage to achieve
significant improvements in both accuracy and frame rate. The code and
pretrained models will be released at
\url{https://github.com/yanglei18/Lite-FPN}.
