  Interpreting camera data is key for autonomously acting systems, such as
autonomous vehicles. Vision systems that operate in real-world environments
must be able to understand their surroundings and need the ability to deal with
novel situations. This paper tackles open-world semantic segmentation, i.e.,
the variant of interpreting image data in which objects occur that have not
been seen during training. We propose a novel approach that performs accurate
closed-world semantic segmentation and, at the same time, can identify new
categories without requiring any additional training data. Our approach
additionally provides a similarity measure for every newly discovered class in
an image to a known category, which can be useful information in downstream
tasks such as planning or mapping. Through extensive experiments, we show that
our model achieves state-of-the-art results on classes known from training data
as well as for anomaly segmentation and can distinguish between different
unknown classes.
