  In the field of autonomous driving, accurate and comprehensive perception of
the 3D environment is crucial. Bird's Eye View (BEV) based methods have emerged
as a promising solution for 3D object detection using multi-view images as
input. However, existing 3D object detection methods often ignore the physical
context in the environment, such as sidewalk and vegetation, resulting in
sub-optimal performance. In this paper, we propose a novel approach called
SOGDet (Semantic-Occupancy Guided Multi-view 3D Object Detection), that
leverages a 3D semantic-occupancy branch to improve the accuracy of 3D object
detection. In particular, the physical context modeled by semantic occupancy
helps the detector to perceive the scenes in a more holistic view. Our SOGDet
is flexible to use and can be seamlessly integrated with most existing
BEV-based methods. To evaluate its effectiveness, we apply this approach to
several state-of-the-art baselines and conduct extensive experiments on the
exclusive nuScenes dataset. Our results show that SOGDet consistently enhance
the performance of three baseline methods in terms of nuScenes Detection Score
(NDS) and mean Average Precision (mAP). This indicates that the combination of
3D object detection and 3D semantic occupancy leads to a more comprehensive
perception of the 3D environment, thereby aiding build more robust autonomous
driving systems. The codes are available at: https://github.com/zhouqiu/SOGDet.
