  Unsupervised Anomaly Detection (UAD) with incremental training is crucial in
industrial manufacturing, as unpredictable defects make obtaining sufficient
labeled data infeasible. However, continual learning methods primarily rely on
supervised annotations, while the application in UAD is limited due to the
absence of supervision. Current UAD methods train separate models for different
classes sequentially, leading to catastrophic forgetting and a heavy
computational burden. To address this issue, we introduce a novel Unsupervised
Continual Anomaly Detection framework called UCAD, which equips the UAD with
continual learning capability through contrastively-learned prompts. In the
proposed UCAD, we design a Continual Prompting Module (CPM) by utilizing a
concise key-prompt-knowledge memory bank to guide task-invariant `anomaly'
model predictions using task-specific `normal' knowledge. Moreover,
Structure-based Contrastive Learning (SCL) is designed with the Segment
Anything Model (SAM) to improve prompt learning and anomaly segmentation
results. Specifically, by treating SAM's masks as structure, we draw features
within the same mask closer and push others apart for general feature
representations. We conduct comprehensive experiments and set the benchmark on
unsupervised continual anomaly detection and segmentation, demonstrating that
our method is significantly better than anomaly detection methods, even with
rehearsal training. The code will be available at
https://github.com/shirowalker/UCAD.
