  There has been a remarkable progress in the accuracy of semantic segmentation
due to the capabilities of deep learning. Unfortunately, these methods are not
able to generalize much further than the distribution of their training data
and fail to handle out-of-distribution classes appropriately. This limits the
applicability to autonomous or safety critical systems. We propose a novel
method leveraging generative models to detect wrongly segmented or
out-of-distribution instances. Conditioned on the predicted semantic
segmentation, an RGB image is generated. We then learn a dissimilarity metric
that compares the generated image with the original input and detects
inconsistencies introduced by the semantic segmentation. We present test cases
for outlier and misclassification detection and evaluate our method
qualitatively and quantitatively on multiple datasets.
