  Mechanical defects in real situations affect observation values and cause
abnormalities in multivariate time series, such as sensor values or network
data. To perceive abnormalities in such data, it is crucial to understand the
temporal context and interrelation between variables simultaneously. The
anomaly detection task for time series, especially for unlabeled data, has been
a challenging problem, and we address it by applying a suitable data
degradation scheme to self-supervised model training. We define four types of
synthetic outliers and propose the degradation scheme in which a portion of
input data is replaced with one of the synthetic outliers. Inspired by the
self-attention mechanism, we design a Transformer-based architecture to
recognize the temporal context and detect unnatural sequences with high
efficiency. Our model converts multivariate data points into temporal
representations with relative position bias and yields anomaly scores from
these representations. Our method, AnomalyBERT, shows a great capability of
detecting anomalies contained in complex time series and surpasses previous
state-of-the-art methods on five real-world benchmarks. Our code is available
at https://github.com/Jhryu30/AnomalyBERT.
