  Software systems log massive amounts of data, recording important runtime
information. Such logs are used, for example, for log-based anomaly detection,
which aims to automatically detect abnormal behaviors of the system under
analysis by processing the information recorded in its logs. Many log-based
anomaly detection techniques based on deep learning models include a
pre-processing step called log parsing. However, understanding the impact of
log parsing on the accuracy of anomaly detection techniques has received
surprisingly little attention so far. Investigating what are the key properties
log parsing techniques should ideally have to help anomaly detection is
therefore warranted.
  In this paper, we report on a comprehensive empirical study on the impact of
log parsing on anomaly detection accuracy, using 13 log parsing techniques,
seven anomaly detection techniques (five based on deep learning and two based
on traditional machine learning) on three publicly available log datasets. Our
empirical results show that, despite what is widely assumed, there is no strong
correlation between log parsing accuracy and anomaly detection accuracy,
regardless of the metric used for measuring log parsing accuracy. Moreover, we
experimentally confirm existing theoretical results showing that it is a
property that we refer to as distinguishability in log parsing results as
opposed to their accuracy that plays an essential role in achieving accurate
anomaly detection.
