  Reliable scene understanding is indispensable for modern autonomous systems.
Current learning-based methods typically try to maximize their performance
based on segmentation metrics that only consider the quality of the
segmentation. However, for the safe operation of a system in the real world it
is crucial to consider the uncertainty in the prediction as well. In this work,
we introduce the novel task of uncertainty-aware panoptic segmentation, which
aims to predict per-pixel semantic and instance segmentations, together with
per-pixel uncertainty estimates. We define two novel metrics to facilitate its
quantitative analysis, the uncertainty-aware Panoptic Quality (uPQ) and the
panoptic Expected Calibration Error (pECE). We further propose the novel
top-down Evidential Panoptic Segmentation Network (EvPSNet) to solve this task.
Our architecture employs a simple yet effective panoptic fusion module that
leverages the predicted uncertainties. Furthermore, we provide several strong
baselines combining state-of-the-art panoptic segmentation networks with
sampling-free uncertainty estimation techniques. Extensive evaluations show
that our EvPSNet achieves the new state-of-the-art for the standard Panoptic
Quality (PQ), as well as for our uncertainty-aware panoptic metrics. We make
the code available at: \url{https://github.com/kshitij3112/EvPSNet}
