  Video anomaly detection is an essential but challenging task. The prevalent
methods mainly investigate the reconstruction difference between normal and
abnormal patterns but ignore the semantics consistency between appearance and
motion information of behavior patterns, making the results highly dependent on
the local context of frame sequences and lacking the understanding of behavior
semantics. To address this issue, we propose a framework of Appearance-Motion
Semantics Representation Consistency that uses the gap of appearance and motion
semantic representation consistency between normal and abnormal data. The
two-stream structure is designed to encode the appearance and motion
information representation of normal samples, and a novel consistency loss is
proposed to enhance the consistency of feature semantics so that anomalies with
low consistency can be identified. Moreover, the lower consistency features of
anomalies can be used to deteriorate the quality of the predicted frame, which
makes anomalies easier to spot. Experimental results demonstrate the
effectiveness of the proposed method.
