  This brief sketches initial progress towards a unified energy-based solution
for the semi-supervised visual anomaly detection and localization problem. In
this setup, we have access to only anomaly-free training data and want to
detect and identify anomalies of an arbitrary nature on test data. We employ
the density estimates from the energy-based model (EBM) as normalcy scores that
can be used to discriminate normal images from anomalous ones. Further, we
back-propagate the gradients of the energy score with respect to the image in
order to generate a gradient map that provides pixel-level spatial localization
of the anomalies in the image. In addition to the spatial localization, we show
that simple processing of the gradient map can also provide alternative
normalcy scores that either match or surpass the detection performance obtained
with the energy value. To quantitatively validate the performance of the
proposed method, we conduct experiments on the MVTec industrial dataset. Though
still preliminary, our results are very promising and reveal the potential of
EBMs for simultaneously detecting and localizing unforeseen anomalies in
images.
