  Multi-view 3D object detection is a crucial component of autonomous driving
systems. Contemporary query-based methods primarily depend either on
dataset-specific initialization of 3D anchors, introducing bias, or utilize
dense attention mechanisms, which are computationally inefficient and
unscalable. To overcome these issues, we present MDHA, a novel sparse
query-based framework, which constructs adaptive 3D output proposals using
hybrid anchors from multi-view, multi-scale input. Fixed 2D anchors are
combined with depth predictions to form 2.5D anchors, which are projected to
obtain 3D proposals. To ensure high efficiency, our proposed Anchor Encoder
performs sparse refinement and selects the top-k anchors and features.
Moreover, while existing multi-view attention mechanisms rely on projecting
reference points to multiple images, our novel Circular Deformable Attention
mechanism only projects to a single image but allows reference points to
seamlessly attend to adjacent images, improving efficiency without compromising
on performance. On the nuScenes val set, it achieves 46.4% mAP and 55.0% NDS
with a ResNet101 backbone. MDHA significantly outperforms the baseline, where
anchor proposals are modelled as learnable embeddings.
