  Anomaly detection is a critical task in industrial manufacturing, aiming to
identify defective parts of products. Most industrial anomaly detection methods
assume the availability of sufficient normal data for training. This assumption
may not hold true due to the cost of labeling or data privacy policies.
Additionally, mainstream methods require training bespoke models for different
objects, which incurs heavy costs and lacks flexibility in practice. To address
these issues, we seek help from Stable Diffusion (SD) model due to its
capability of zero/few-shot inpainting, which can be leveraged to inpaint
anomalous regions as normal. In this paper, a few-shot multi-class anomaly
detection framework that adopts Stable Diffusion model is proposed, named
AnomalySD. To adapt SD to anomaly detection task, we design different
hierarchical text descriptions and the foreground mask mechanism for
fine-tuning SD. In the inference stage, to accurately mask anomalous regions
for inpainting, we propose multi-scale mask strategy and prototype-guided mask
strategy to handle diverse anomalous regions. Hierarchical text prompts are
also utilized to guide the process of inpainting in the inference stage. The
anomaly score is estimated based on inpainting result of all masks. Extensive
experiments on the MVTec-AD and VisA datasets demonstrate the superiority of
our approach. We achieved anomaly classification and segmentation results of
93.6%/94.8% AUROC on the MVTec-AD dataset and 86.1%/96.5% AUROC on the VisA
dataset under multi-class and one-shot settings.
