  Continued improvements in deep learning architectures have steadily advanced
the overall performance of 3D object detectors to levels on par with humans for
certain tasks and datasets, where the overall performance is mostly driven by
common examples. However, even the best performing models suffer from the most
naive mistakes when it comes to rare examples that do not appear frequently in
the training data, such as vehicles with irregular geometries. Most studies in
the long-tail literature focus on class-imbalanced classification problems with
known imbalanced label counts per class, but they are not directly applicable
to the intra-class long-tail examples in problems with large intra-class
variations such as 3D object detection, where instances with the same class
label can have drastically varied properties such as shapes and sizes. Other
works propose to mitigate this problem using active learning based on the
criteria of uncertainty, difficulty, or diversity. In this study, we identify a
new conceptual dimension - rareness - to mine new data for improving the
long-tail performance of models. We show that rareness, as opposed to
difficulty, is the key to data-centric improvements for 3D detectors, since
rareness is the result of a lack in data support while difficulty is related to
the fundamental ambiguity in the problem. We propose a general and effective
method to identify the rareness of objects based on density estimation in the
feature space using flow models, and propose a principled cost-aware
formulation for mining rare object tracks, which improves overall model
performance, but more importantly - significantly improves the performance for
rare objects (by 30.97\%
