  Semantic segmentation is an important task that helps autonomous vehicles
understand their surroundings and navigate safely. During deployment, even the
most mature segmentation models are vulnerable to various external factors that
can degrade the segmentation performance with potentially catastrophic
consequences for the vehicle and its surroundings. To address this issue, we
propose a failure detection framework to identify pixel-level
misclassification. We do so by exploiting internal features of the segmentation
model and training it simultaneously with a failure detection network. During
deployment, the failure detector can flag areas in the image where the
segmentation model have failed to segment correctly. We evaluate the proposed
approach against state-of-the-art methods and achieve 12.30%, 9.46%, and 9.65%
performance improvement in the AUPR-Error metric for Cityscapes, BDD100K, and
Mapillary semantic segmentation datasets.
