  Multi-camera 3D object detection for autonomous driving is a challenging
problem that has garnered notable attention from both academia and industry. An
obstacle encountered in vision-based techniques involves the precise extraction
of geometry-conscious features from RGB images. Recent approaches have utilized
geometric-aware image backbones pretrained on depth-relevant tasks to acquire
spatial information. However, these approaches overlook the critical aspect of
view transformation, resulting in inadequate performance due to the
misalignment of spatial knowledge between the image backbone and view
transformation. To address this issue, we propose a novel geometric-aware
pretraining framework called GAPretrain. Our approach incorporates spatial and
structural cues to camera networks by employing the geometric-rich modality as
guidance during the pretraining phase. The transference of modal-specific
attributes across different modalities is non-trivial, but we bridge this gap
by using a unified bird's-eye-view (BEV) representation and structural hints
derived from LiDAR point clouds to facilitate the pretraining process.
GAPretrain serves as a plug-and-play solution that can be flexibly applied to
multiple state-of-the-art detectors. Our experiments demonstrate the
effectiveness and generalization ability of the proposed method. We achieve
46.2 mAP and 55.5 NDS on the nuScenes val set using the BEVFormer method, with
a gain of 2.7 and 2.1 points, respectively. We also conduct experiments on
various image backbones and view transformations to validate the efficacy of
our approach. Code will be released at
https://github.com/OpenDriveLab/BEVPerception-Survey-Recipe.
