  Anomaly detection, where data instances are discovered containing feature
patterns different from the majority, plays a fundamental role in various
applications. However, it is challenging for existing methods to handle the
scenarios where the instances are systems whose characteristics are not readily
observed as data. Appropriate interactions are needed to interact with the
systems and identify those with abnormal responses. Detecting system-wise
anomalies is a challenging task due to several reasons including: how to
formally define the system-wise anomaly detection problem; how to find the
effective activation signal for interacting with systems to progressively
collect the data and learn the detector; how to guarantee stable training in
such a non-stationary scenario with real-time interactions? To address the
challenges, we propose InterSAD (Interactive System-wise Anomaly Detection).
Specifically, first, we adopt Markov decision process to model the interactive
systems, and define anomalous systems as anomalous transition and anomalous
reward systems. Then, we develop an end-to-end approach which includes an
encoder-decoder module that learns system embeddings, and a policy network to
generate effective activation for separating embeddings of normal and anomaly
systems. Finally, we design a training method to stabilize the learning
process, which includes a replay buffer to store historical interaction data
and allow them to be re-sampled. Experiments on two benchmark environments,
including identifying the anomalous robotic systems and detecting user data
poisoning in recommendation models, demonstrate the superiority of InterSAD
compared with state-of-the-art baselines methods.
