  We introduce Probabilistic Object Detection, the task of detecting objects in
images and accurately quantifying the spatial and semantic uncertainties of the
detections. Given the lack of methods capable of assessing such probabilistic
object detections, we present the new Probability-based Detection Quality
measure (PDQ).Unlike AP-based measures, PDQ has no arbitrary thresholds and
rewards spatial and label quality, and foreground/background separation quality
while explicitly penalising false positive and false negative detections. We
contrast PDQ with existing mAP and moLRP measures by evaluating
state-of-the-art detectors and a Bayesian object detector based on Monte Carlo
Dropout. Our experiments indicate that conventional object detectors tend to be
spatially overconfident and thus perform poorly on the task of probabilistic
object detection. Our paper aims to encourage the development of new object
detection approaches that provide detections with accurately estimated spatial
and label uncertainties and are of critical importance for deployment on robots
and embodied AI systems in the real world.
