  In recent years, a number of process-based anomaly detection schemes for
Industrial Control Systems were proposed. In this work, we provide the first
systematic analysis of such schemes, and introduce a taxonomy of properties
that are verified by those detection systems. We then present a novel general
framework to generate adversarial spoofing signals that violate physical
properties of the system, and use the framework to analyze four anomaly
detectors published at top security conferences. We find that three of those
detectors are susceptible to a number of adversarial manipulations (e.g.,
spoofing with precomputed patterns), which we call Synthetic Sensor Spoofing
and one is resilient against our attacks. We investigate the root of its
resilience and demonstrate that it comes from the properties that we
introduced. Our attacks reduce the Recall (True Positive Rate) of the attacked
schemes making them not able to correctly detect anomalies. Thus, the
vulnerabilities we discovered in the anomaly detectors show that (despite an
original good detection performance), those detectors are not able to reliably
learn physical properties of the system. Even attacks that prior work was
expected to be resilient against (based on verified properties) were found to
be successful. We argue that our findings demonstrate the need for both more
complete attacks in datasets, and more critical analysis of process-based
anomaly detectors. We plan to release our implementation as open-source,
together with an extension of two public datasets with a set of Synthetic
Sensor Spoofing attacks as generated by our framework.
