  The availability of real-world datasets is the prerequisite for developing
object detection methods for autonomous driving. While ambiguity exists in
object labels due to error-prone annotation process or sensor observation
noises, current object detection datasets only provide deterministic
annotations without considering their uncertainty. This precludes an in-depth
evaluation among different object detection methods, especially for those that
explicitly model predictive probability. In this work, we propose a generative
model to estimate bounding box label uncertainties from LiDAR point clouds, and
define a new representation of the probabilistic bounding box through spatial
distribution. Comprehensive experiments show that the proposed model represents
uncertainties commonly seen in driving scenarios. Based on the spatial
distribution, we further propose an extension of IoU, called the Jaccard IoU
(JIoU), as a new evaluation metric that incorporates label uncertainty.
Experiments on the KITTI and the Waymo Open Datasets show that JIoU is superior
to IoU when evaluating probabilistic object detectors.
