  Logs are widely used in the development and maintenance of software systems.
Logs can help engineers understand the runtime behavior of systems and diagnose
system failures. For anomaly diagnosis, existing methods generally use log
event data extracted from historical logs to build diagnostic models. However,
we find that existing methods do not make full use of two types of features,
(1) statistical features: some inherent statistical features in log data, such
as word frequency and abnormal label distribution, are not well exploited.
Compared with log raw data, statistical features are deterministic and
naturally compatible with corresponding tasks. (2) semantic features: Logs
contain the execution logic behind software systems, thus log statements share
deep semantic relationships. How to effectively combine statistical features
and semantic features in log data to improve the performance of log anomaly
diagnosis is the key point of this paper. In this paper, we propose an adaptive
semantic gate networks (ASGNet) that combines statistical features and semantic
features to selectively use statistical features to consolidate log text
semantic representation. Specifically, ASGNet encodes statistical features via
a variational encoding module and fuses useful information through a
well-designed adaptive semantic threshold mechanism. The threshold mechanism
introduces the information flow into the classifier based on the confidence of
the semantic features in the decision, which is conducive to training a robust
classifier and can solve the overfitting problem caused by the use of
statistical features. The experimental results on the real data set show that
our method proposed is superior to all baseline methods in terms of various
performance indicators.
