  Contrastive Language-Image Pre-training (CLIP) models have shown promising
performance on zero-shot visual recognition tasks by learning visual
representations under natural language supervision. Recent studies attempt the
use of CLIP to tackle zero-shot anomaly detection by matching images with
normal and abnormal state prompts. However, since CLIP focuses on building
correspondence between paired text prompts and global image-level
representations, the lack of fine-grained patch-level vision to text alignment
limits its capability on precise visual anomaly localization. In this work, we
propose AnoCLIP for zero-shot anomaly localization. In the visual encoder, we
introduce a training-free value-wise attention mechanism to extract intrinsic
local tokens of CLIP for patch-level local description. From the perspective of
text supervision, we particularly design a unified domain-aware contrastive
state prompting template for fine-grained vision-language matching. On top of
the proposed AnoCLIP, we further introduce a test-time adaptation (TTA)
mechanism to refine visual anomaly localization results, where we optimize a
lightweight adapter in the visual encoder using AnoCLIP's pseudo-labels and
noise-corrupted tokens. With both AnoCLIP and TTA, we significantly exploit the
potential of CLIP for zero-shot anomaly localization and demonstrate the
effectiveness of AnoCLIP on various datasets.
