  Due to the complexity and size of modern software systems, the amount of logs
generated is tremendous. Hence, it is infeasible to manually investigate these
data in a reasonable time, thereby requiring automating log analysis to derive
insights about the functioning of the systems. Motivated by an industry
use-case, we zoom-in on one integral part of automated log analysis, log
parsing, which is the prerequisite to deriving any insights from logs. Our
investigation reveals problematic aspects within the log parsing field,
particularly its inefficiency in handling heterogeneous real-world logs. We
show this by assessing the 14 most-recognized log parsing approaches in the
literature using (i) nine publicly available datasets, (ii) one dataset
comprised of combined publicly available data, and (iii) one dataset generated
within the infrastructure of a large bank. Subsequently, toward improving log
parsing robustness in real-world production scenarios, we propose a tool,
Logchimera, that enables estimating log parsing performance in industry
contexts through generating synthetic log data that resemble industry logs. Our
contributions serve as a foundation to consolidate past research efforts,
facilitate future research advancements, and establish a strong link between
research and industry log parsing.
