  Classifying samples as in-distribution or out-of-distribution (OOD) is a
challenging problem of anomaly detection and a strong test of the
generalisation power for models of the in-distribution. In this paper, we
present a simple and generic framework, {\it SemSAD}, that makes use of a
semantic similarity score to carry out anomaly detection. The idea is to first
find for any test example the semantically closest examples in the training
set, where the semantic relation between examples is quantified by the cosine
similarity between feature vectors that leave semantics unchanged under
transformations, such as geometric transformations (images), time shifts (audio
signals), and synonymous word substitutions (text). A trained discriminator is
then used to classify a test example as OOD if the semantic similarity to its
nearest neighbours is significantly lower than the corresponding similarity for
test examples from the in-distribution. We are able to outperform previous
approaches for anomaly, novelty, or out-of-distribution detection in the visual
domain by a large margin. In particular, we obtain AUROC values close to one
for the challenging task of detecting examples from CIFAR-10 as
out-of-distribution given CIFAR-100 as in-distribution, without making use of
label information.
