  The deployment of 3D detectors strikes one of the major challenges in
real-world self-driving scenarios. Existing BEV-based (i.e., Bird Eye View)
detectors favor sparse convolutions (known as SPConv) to speed up training and
inference, which puts a hard barrier for deployment, especially for on-device
applications. In this paper, to tackle the challenge of efficient 3D object
detection from an industry perspective, we devise a deployment-friendly
pillar-based 3D detector, termed FastPillars. First, we introduce a novel
lightweight Max-and-Attention Pillar Encoding (MAPE) module specially for
enhancing small 3D objects. Second, we propose a simple yet effective principle
for designing a backbone in pillar-based 3D detection. We construct FastPillars
based on these designs, achieving high performance and low latency without
SPConv. Extensive experiments on two large-scale datasets demonstrate the
effectiveness and efficiency of FastPillars for on-device 3D detection
regarding both performance and speed. Specifically, FastPillars delivers
state-of-the-art accuracy on Waymo Open Dataset with 1.8X speed up and 3.8
mAPH/L2 improvement over CenterPoint (SPConv-based). Our code is publicly
available at: https://github.com/StiphyJay/FastPillars.
