  Frame reconstruction (current or future frame) based on Auto-Encoder (AE) is
a popular method for video anomaly detection. With models trained on the normal
data, the reconstruction errors of anomalous scenes are usually much larger
than those of normal ones. Previous methods introduced the memory bank into AE,
for encoding diverse normal patterns across the training videos. However, they
are memory consuming and cannot cope with unseen new scenarios in the testing
data. In this work, we propose a self-attention prototype unit (APU) to encode
the normal latent space as prototypes in real time, free from extra memory
cost. In addition, we introduce circulative attention mechanism to our backbone
to form a novel feature extracting learner, namely Circulative Attention Unit
(CAU). It enables the fast adaption capability on new scenes by only consuming
a few iterations of update. Extensive experiments are conducted on various
benchmarks. The superior performance over the state-of-the-art demonstrates the
effectiveness of our method. Our code is available at
https://github.com/huchao-AI/APN/.
