  In this paper, we attack the anomaly detection problem by directly modeling
the data distribution with deep architectures. We propose deep structured
energy based models (DSEBMs), where the energy function is the output of a
deterministic deep neural network with structure. We develop novel model
architectures to integrate EBMs with different types of data such as static
data, sequential data, and spatial data, and apply appropriate model
architectures to adapt to the data structure. Our training algorithm is built
upon the recent development of score matching \cite{sm}, which connects an EBM
with a regularized autoencoder, eliminating the need for complicated sampling
method. Statistically sound decision criterion can be derived for anomaly
detection purpose from the perspective of the energy landscape of the data
distribution. We investigate two decision criteria for performing anomaly
detection: the energy score and the reconstruction error. Extensive empirical
studies on benchmark tasks demonstrate that our proposed model consistently
matches or outperforms all the competing methods.
