  3D object detection using LiDAR data is an indispensable component for
autonomous driving systems. Yet, only a few LiDAR-based 3D object detection
methods leverage segmentation information to further guide the detection
process. In this paper, we propose a novel multi-task framework that jointly
performs 3D object detection and panoptic segmentation. In our method, the 3D
object detection backbone in Bird's-Eye-View (BEV) plane is augmented by the
injection of Range-View (RV) feature maps from the 3D panoptic segmentation
backbone. This enables the detection backbone to leverage multi-view
information to address the shortcomings of each projection view. Furthermore,
foreground semantic information is incorporated to ease the detection task by
highlighting the locations of each object class in the feature maps. Finally, a
new center density heatmap generated based on the instance-level information
further guides the detection backbone by suggesting possible box center
locations for objects. Our method works with any BEV-based 3D object detection
method, and as shown by extensive experiments on the nuScenes dataset, it
provides significant performance gains. Notably, the proposed method based on a
single-stage CenterPoint 3D object detection network achieved state-of-the-art
performance on nuScenes 3D Detection Benchmark with 67.3 NDS.
