  In recent years, there is an increasing interests in reconstruction based
generative models for image One-Class Novelty Detection, most of which only
focus on image-level information. While in this paper, we further exploit the
latent space of Variational Auto-encoder (VAE), a typical reconstruction based
model, and we innovatively divide it into three regions:
Normal/Anomalous/Unknown-semantic-region. Based on this hypothesis, we propose
a new VAE architecture, Recoding Semantic Consistency Based VAE (RSC-VAE),
combining VAE with recoding mechanism and constraining the semantic consistency
of two encodings. We come up with three training modes of RSC-VAE: 1. One-Class
Training Mode, alleviating False Positive problem of normal samples; 2.
Distributionally-Shifted Training Mode, alleviating False Negative problem of
anomalous samples; 3. Extremely-Imbalanced Training Mode, introducing a small
number of anomalous samples for training to enhance the second mode. The
experimental results on multiple datasets demonstrate that our mechanism
achieves state-of-the-art performance in various baselines including VAE.
