  Considerable research effort has been devoted to LiDAR-based 3D object
detection and empirical performance has been significantly improved. While
progress has been encouraging, we observe an overlooked issue: it is not yet
common practice to compare different 3D detectors under the same cost, e.g.,
inference latency. This makes it difficult to quantify the true performance
gain brought by recently proposed architecture designs. The goal of this work
is to conduct a cost-aware evaluation of LiDAR-based 3D object detectors.
Specifically, we focus on SECOND, a simple grid-based one-stage detector, and
analyze its performance under different costs by scaling its original
architecture. Then we compare the family of scaled SECOND with recent 3D
detection methods, such as Voxel R-CNN and PV-RCNN++. The results are
surprising. We find that, if allowed to use the same latency, SECOND can match
the performance of PV-RCNN++, the current state-of-the-art method on the Waymo
Open Dataset. Scaled SECOND also easily outperforms many recent 3D detection
methods published during the past year. We recommend future research control
the inference cost in their empirical comparison and include the family of
scaled SECOND as a strong baseline when presenting novel 3D detection methods.
