  Camera and LiDAR sensor modalities provide complementary appearance and
geometric information useful for detecting 3D objects for autonomous vehicle
applications. However, current end-to-end fusion methods are challenging to
train and underperform state-of-the-art LiDAR-only detectors. Sequential fusion
methods suffer from a limited number of pixel and point correspondences due to
point cloud sparsity, or their performance is strictly capped by the detections
of one of the modalities. Our proposed solution, Dense Voxel Fusion (DVF) is a
sequential fusion method that generates multi-scale dense voxel feature
representations, improving expressiveness in low point density regions. To
enhance multi-modal learning, we train directly with projected ground truth 3D
bounding box labels, avoiding noisy, detector-specific 2D predictions. Both DVF
and the multi-modal training approach can be applied to any voxel-based LiDAR
backbone. DVF ranks 3rd among published fusion methods on KITTI 3D car
detection benchmark without introducing additional trainable parameters, nor
requiring stereo images or dense depth labels. In addition, DVF significantly
improves 3D vehicle detection performance of voxel-based methods on the Waymo
Open Dataset.
