  With the escalating threats posed by cyberattacks on Industrial Control
Systems (ICSs), the development of customized Industrial Intrusion Detection
Systems (IIDSs) received significant attention in research. While existing
literature proposes effective IIDS solutions evaluated in controlled
environments, their deployment in real-world industrial settings poses several
challenges. This paper highlights two critical yet often overlooked aspects
that significantly impact their practical deployment, i.e., the need for
sufficient amounts of data to train the IIDS models and the challenges
associated with finding suitable hyperparameters, especially for IIDSs training
only on genuine ICS data. Through empirical experiments conducted on multiple
state-of-the-art IIDSs and diverse datasets, we establish the criticality of
these issues in deploying IIDSs. Our findings show the necessity of extensive
malicious training data for supervised IIDSs, which can be impractical
considering the complexity of recording and labeling attacks in actual
industrial environments. Furthermore, while other IIDSs circumvent the previous
issue by requiring only benign training data, these can suffer from the
difficulty of setting appropriate hyperparameters, which likewise can diminish
their performance. By shedding light on these challenges, we aim to enhance the
understanding of the limitations and considerations necessary for deploying
effective cybersecurity solutions in ICSs, which might be one reason why IIDSs
see few deployments.
