  The increasing complexity of modern high-performance computing (HPC) systems
necessitates the introduction of automated and data-driven methodologies to
support system administrators' effort toward increasing the system's
availability. Anomaly detection is an integral part of improving the
availability as it eases the system administrator's burden and reduces the time
between an anomaly and its resolution. However, current state-of-the-art (SoA)
approaches to anomaly detection are supervised and semi-supervised, so they
require a human-labelled dataset with anomalies - this is often impractical to
collect in production HPC systems. Unsupervised anomaly detection approaches
based on clustering, aimed at alleviating the need for accurate anomaly data,
have so far shown poor performance.
  In this work, we overcome these limitations by proposing RUAD, a novel
Recurrent Unsupervised Anomaly Detection model. RUAD achieves better results
than the current semi-supervised and unsupervised SoA approaches. This is
achieved by considering temporal dependencies in the data and including
long-short term memory cells in the model architecture. The proposed approach
is assessed on a complete ten-month history of a Tier-0 system (Marconi100 from
CINECA with 980 nodes). RUAD achieves an area under the curve (AUC) of 0.763 in
semi-supervised training and an AUC of 0.767 in unsupervised training, which
improves upon the SoA approach that achieves an AUC of 0.747 in semi-supervised
training and an AUC of 0.734 in unsupervised training. It also vastly
outperforms the current SoA unsupervised anomaly detection approach based on
clustering, achieving the AUC of 0.548.
