  This paper presents Multi-view Labelling Object Detector (MLOD). The detector
takes an RGB image and a LIDAR point cloud as input and follows the two-stage
object detection framework. A Region Proposal Network (RPN) generates 3D
proposals in a Bird's Eye View (BEV) projection of the point cloud. The second
stage projects the 3D proposal bounding boxes to the image and BEV feature maps
and sends the corresponding map crops to a detection header for classification
and bounding-box regression. Unlike other multi-view based methods, the cropped
image features are not directly fed to the detection header, but masked by the
depth information to filter out parts outside 3D bounding boxes. The fusion of
image and BEV features is challenging, as they are derived from different
perspectives. We introduce a novel detection header, which provides detection
results not just from fusion layer, but also from each sensor channel. Hence
the object detector can be trained on data labelled in different views to avoid
the degeneration of feature extractors. MLOD achieves state-of-the-art
performance on the KITTI 3D object detection benchmark. Most importantly, the
evaluation shows that the new header architecture is effective in preventing
image feature extractor degeneration.
