  Recent efforts towards video anomaly detection (VAD) try to learn a deep
autoencoder to describe normal event patterns with small reconstruction errors.
The video inputs with large reconstruction errors are regarded as anomalies at
the test time. However, these methods sometimes reconstruct abnormal inputs
well because of the powerful generalization ability of deep autoencoder. To
address this problem, we present a novel approach for anomaly detection, which
utilizes discriminative prototypes of normal data to reconstruct video frames.
In this way, the model will favor the reconstruction of normal events and
distort the reconstruction of abnormal events. Specifically, we use a
prototype-guided memory module to perform discriminative latent embedding. We
introduce a new discriminative criterion for the memory module, as well as a
loss function correspondingly, which can encourage memory items to record the
representative embeddings of normal data, i.e. prototypes. Besides, we design a
novel two-branch autoencoder, which is composed of a future frame prediction
network and an RGB difference generation network that share the same encoder.
The stacked RGB difference contains motion information just like optical flow,
so our model can learn temporal regularity. We evaluate the effectiveness of
our method on three benchmark datasets and experimental results demonstrate the
proposed method outperforms the state-of-the-art.
