  The key to out-of-distribution detection is density estimation of the
in-distribution data or of its feature representations. This is particularly
challenging for dense anomaly detection in domains where the in-distribution
data has a complex underlying structure. Nearest-Neighbors approaches have been
shown to work well in object-centric data domains, such as industrial
inspection and image classification. In this paper, we show that
nearest-neighbor approaches also yield state-of-the-art results on dense
novelty detection in complex driving scenes when working with an appropriate
feature representation. In particular, we find that transformer-based
architectures produce representations that yield much better similarity metrics
for the task. We identify the multi-head structure of these models as one of
the reasons, and demonstrate a way to transfer some of the improvements to
CNNs. Ultimately, the approach is simple and non-invasive, i.e., it does not
affect the primary segmentation performance, refrains from training on examples
of anomalies, and achieves state-of-the-art results on RoadAnomaly,
StreetHazards, and SegmentMeIfYouCan-Anomaly.
