  Deep learning has recently demonstrated state-of-the art performance on key
tasks related to the maintenance of computer systems, such as intrusion
detection, denial of service attack detection, hardware and software system
failures, and malware detection. In these contexts, model interpretability is
vital for administrator and analyst to trust and act on the automated analysis
of machine learning models. Deep learning methods have been criticized as black
box oracles which allow limited insight into decision factors. In this work we
seek to "bridge the gap" between the impressive performance of deep learning
models and the need for interpretable model introspection. To this end we
present recurrent neural network (RNN) language models augmented with attention
for anomaly detection in system logs. Our methods are generally applicable to
any computer system and logging source.
  By incorporating attention variants into our RNN language models we create
opportunities for model introspection and analysis without sacrificing
state-of-the art performance.
  We demonstrate model performance and illustrate model interpretability on an
intrusion detection task using the Los Alamos National Laboratory (LANL) cyber
security dataset, reporting upward of 0.99 area under the receiver operator
characteristic curve despite being trained only on a single day's worth of
data.
