  3D object detection from multiple image views is a fundamental and
challenging task for visual scene understanding. Owing to its low cost and high
efficiency, multi-view 3D object detection has demonstrated promising
application prospects. However, accurately detecting objects through
perspective views is extremely difficult due to the lack of depth information.
Current approaches tend to adopt heavy backbones for image encoders, making
them inapplicable for real-world deployment. Different from the images, LiDAR
points are superior in providing spatial cues, resulting in highly precise
localization. In this paper, we explore the incorporation of LiDAR-based
detectors for multi-view 3D object detection. Instead of directly training a
depth prediction network, we unify the image and LiDAR features in the
Bird-Eye-View (BEV) space and adaptively transfer knowledge across
non-homogenous representations in a teacher-student paradigm. To this end, we
propose \textbf{BEVDistill}, a cross-modal BEV knowledge distillation (KD)
framework for multi-view 3D object detection. Extensive experiments demonstrate
that the proposed method outperforms current KD approaches on a
highly-competitive baseline, BEVFormer, without introducing any extra cost in
the inference phase. Notably, our best model achieves 59.4 NDS on the nuScenes
test leaderboard, achieving new state-of-the-art in comparison with various
image-based detectors. Code will be available at
https://github.com/zehuichen123/BEVDistill.
