  This paper studies zero-shot anomaly classification (AC) and segmentation
(AS) in industrial vision. We reveal that the abundant normal and abnormal cues
implicit in unlabeled test images can be exploited for anomaly determination,
which is ignored by prior methods. Our key observation is that for the
industrial product images, the normal image patches could find a relatively
large number of similar patches in other unlabeled images, while the abnormal
ones only have a few similar patches. We leverage such a discriminative
characteristic to design a novel zero-shot AC/AS method by Mutual Scoring
(MuSc) of the unlabeled images, which does not need any training or prompts.
Specifically, we perform Local Neighborhood Aggregation with Multiple Degrees
(LNAMD) to obtain the patch features that are capable of representing anomalies
in varying sizes. Then we propose the Mutual Scoring Mechanism (MSM) to
leverage the unlabeled test images to assign the anomaly score to each other.
Furthermore, we present an optimization approach named Re-scoring with
Constrained Image-level Neighborhood (RsCIN) for image-level anomaly
classification to suppress the false positives caused by noises in normal
images. The superior performance on the challenging MVTec AD and VisA datasets
demonstrates the effectiveness of our approach. Compared with the
state-of-the-art zero-shot approaches, MuSc achieves a $\textbf{21.1%}$ PRO
absolute gain (from 72.7% to 93.8%) on MVTec AD, a $\textbf{19.4%}$ pixel-AP
gain and a $\textbf{14.7%}$ pixel-AUROC gain on VisA. In addition, our
zero-shot approach outperforms most of the few-shot approaches and is
comparable to some one-class methods. Code is available at
https://github.com/xrli-U/MuSc.
