  Anomaly detection for time-series data becomes an essential task for many
data-driven applications fueled with an abundance of data and out-of-the-box
machine-learning algorithms. In many real-world settings, developing a reliable
anomaly model is highly challenging due to insufficient anomaly labels and the
prohibitively expensive cost of obtaining anomaly examples. It imposes a
significant bottleneck to evaluate model quality for model selection and
parameter tuning reliably. As a result, many existing anomaly detection
algorithms fail to show their promised performance after deployment.
  In this paper, we propose LaF-AD, a novel anomaly detection algorithm with
label-free model selection for unlabeled times-series data. Our proposed
algorithm performs a fully unsupervised ensemble learning across a large number
of candidate parametric models. We develop a model variance metric that
quantifies the sensitivity of anomaly probability with a bootstrapping method.
Then it makes a collective decision for anomaly events by model learners using
the model variance. Our algorithm is easily parallelizable, more robust for
ill-conditioned and seasonal data, and highly scalable for a large number of
anomaly models. We evaluate our algorithm against other state-of-the-art
methods on a synthetic domain and a benchmark public data set.
