  In large IT systems, software deployment is a crucial process in online
services as their code is regularly updated. However, a faulty code change may
degrade the target service's performance and cause cascading outages in
downstream services. Thus, software deployments should be comprehensively
monitored, and their anomalies should be detected timely. In this paper, we
study the problem of anomaly detection for deployments. We begin by identifying
the challenges unique to this anomaly detection problem, which is at
entity-level (e.g., deployments), relative to the more typical problem of
anomaly detection in multivariate time series (MTS). The unique challenges
include the heterogeneity of deployments, the low latency tolerance, the
ambiguous anomaly definition, and the limited supervision. To address them, we
propose a novel framework, semi-supervised hybrid Model for Entity-Level Online
Detection of anomalY (MELODY). MELODY first transforms the MTS of different
entities to the same feature space by an online feature extractor, then uses a
newly proposed semi-supervised deep one-class model for detecting anomalous
entities. We evaluated MELODY on real data of cloud services with 1.2M+ time
series. The relative F1 score improvement of MELODY over the state-of-the-art
methods ranges from 7.6% to 56.5%. The user evaluation suggests MELODY is
suitable for monitoring deployments in large online systems.
