  One pivot challenge for image anomaly (AD) detection is to learn
discriminative information only from normal class training images. Most image
reconstruction based AD methods rely on the discriminative capability of
reconstruction error. This is heuristic as image reconstruction is unsupervised
without incorporating normal-class-specific information. In this paper, we
propose an AD method called dual deep reconstruction networks based image
decomposition (DDR-ID). The networks are trained by jointly optimizing for
three losses: the one-class loss, the latent space constrain loss and the
reconstruction loss. After training, DDR-ID can decompose an unseen image into
its normal class and the residual components, respectively. Two anomaly scores
are calculated to quantify the anomalous degree of the image in either normal
class latent space or reconstruction image space. Thereby, anomaly detection
can be performed via thresholding the anomaly score. The experiments
demonstrate that DDR-ID outperforms multiple related benchmarking methods in
image anomaly detection using MNIST, CIFAR-10 and Endosome datasets and
adversarial attack detection using GTSRB dataset.
