  We present a flexible and high-performance framework, named Pyramid R-CNN,
for two-stage 3D object detection from point clouds. Current approaches
generally rely on the points or voxels of interest for RoI feature extraction
on the second stage, but cannot effectively handle the sparsity and non-uniform
distribution of those points, and this may result in failures in detecting
objects that are far away. To resolve the problems, we propose a novel
second-stage module, named pyramid RoI head, to adaptively learn the features
from the sparse points of interest. The pyramid RoI head consists of three key
components. Firstly, we propose the RoI-grid Pyramid, which mitigates the
sparsity problem by extensively collecting points of interest for each RoI in a
pyramid manner. Secondly, we propose RoI-grid Attention, a new operation that
can encode richer information from sparse points by incorporating conventional
attention-based and graph-based point operators into a unified formulation.
Thirdly, we propose the Density-Aware Radius Prediction (DARP) module, which
can adapt to different point density levels by dynamically adjusting the
focusing range of RoIs. Combining the three components, our pyramid RoI head is
robust to the sparse and imbalanced circumstances, and can be applied upon
various 3D backbones to consistently boost the detection performance. Extensive
experiments show that Pyramid R-CNN outperforms the state-of-the-art 3D
detection models by a large margin on both the KITTI dataset and the Waymo Open
dataset.
