  This paper shows the effectiveness of 2D backbone scaling and pretraining for
pillar-based 3D object detectors. Pillar-based methods mainly employ randomly
initialized 2D convolution neural network (ConvNet) for feature extraction and
fail to enjoy the benefits from the backbone scaling and pretraining in the
image domain. To show the scaling-up capacity in point clouds, we introduce the
dense ConvNet pretrained on large-scale image datasets (e.g., ImageNet) as the
2D backbone of pillar-based detectors. The ConvNets are adaptively designed
based on the model size according to the specific features of point clouds,
such as sparsity and irregularity. Equipped with the pretrained ConvNets, our
proposed pillar-based detector, termed PillarNeSt, outperforms the existing 3D
object detectors by a large margin on the nuScenes and Argoversev2 datasets.
Our code shall be released upon acceptance.
