  Detecting rare events is essential in various fields, e.g., in cyber security
or maintenance. Often, human experts are supported by anomaly detection systems
as continuously monitoring the data is an error-prone and tedious task.
However, among the anomalies detected may be events that are rare, e.g., a
planned shutdown of a machine, but are not the actual event of interest, e.g.,
breakdowns of a machine. Therefore, human experts are needed to validate
whether the detected anomalies are relevant. We propose to support this anomaly
investigation by providing explanations of anomaly detection. Related work only
focuses on the technical implementation of explainable anomaly detection and
neglects the subsequent human anomaly investigation. To address this research
gap, we conduct a behavioral experiment using records of taxi rides in New York
City as a testbed. Participants are asked to differentiate extreme weather
events from other anomalous events such as holidays or sporting events. Our
results show that providing counterfactual explanations do improve the
investigation of anomalies, indicating potential for explainable anomaly
detection in general.
