  The complexity and scale of IT systems are increasing dramatically, posing
many challenges to real-world anomaly detection. Deep learning anomaly
detection has emerged, aiming at feature learning and anomaly scoring, which
has gained tremendous success. However, little work has been done on the
thresholding problem despite it being a critical factor for the effectiveness
of anomaly detection. In this paper, we model thresholding in anomaly detection
as a Markov Decision Process and propose an agent-based dynamic thresholding
(ADT) framework based on a deep Q-network. The proposed method can be
integrated into many systems that require dynamic thresholding. An auto-encoder
is utilized in this study to obtain feature representations and produce anomaly
scores for complex input data. ADT can adjust thresholds adaptively by
utilizing the anomaly scores from the auto-encoder and significantly improve
anomaly detection performance. The properties of ADT are studied through
experiments on three real-world datasets and compared with benchmarks, hence
demonstrating its thresholding capability, data-efficient learning, stability,
and robustness. Our study validates the effectiveness of reinforcement learning
in optimal thresholding control in anomaly detection.
