  Growth in system complexity increases the need for automated techniques
dedicated to different log analysis tasks such as Log-based Anomaly Detection
(LAD). The latter has been widely addressed in the literature, mostly by means
of a variety of deep learning techniques.
  Despite their many advantages, that focus on deep learning techniques is
somewhat arbitrary as traditional Machine Learning (ML) techniques may perform
well in many cases, depending on the context and datasets. In the same vein,
semi-supervised techniques deserve the same attention as supervised techniques
since the former have clear practical advantages. Further, current evaluations
mostly rely on the assessment of detection accuracy. However, this is not
enough to decide whether or not a specific ML technique is suitable to address
the LAD problem in a given context. Other aspects to consider include training
and prediction times as well as the sensitivity to hyperparameter tuning, which
in practice matters to engineers. In this paper, we present a comprehensive
empirical study, in which we evaluate supervised and semi-supervised,
traditional and deep ML techniques w.r.t. four evaluation criteria: detection
accuracy, time performance, sensitivity of detection accuracy and time
performance to hyperparameter tuning. The experimental results show that
supervised traditional and deep ML techniques fare similarly in terms of their
detection accuracy and prediction time. Moreover, overall, sensitivity analysis
to hyperparameter tuning w.r.t. detection accuracy shows that supervised
traditional ML techniques are less sensitive than deep learning techniques.
Further, semi-supervised techniques yield significantly worse detection
accuracy than supervised techniques.
