  Shortcut learning, or `Clever Hans effect` refers to situations where a
learning agent (e.g., deep neural networks) learns spurious correlations
present in data, resulting in biased models. We focus on finding shortcuts in
deep learning based spoofing countermeasures (CMs) that predict whether a given
utterance is spoofed or not. While prior work has addressed specific data
artifacts, such as silence, no general normative framework has been explored
for analyzing shortcut learning in CMs. In this study, we propose a generic
approach to identifying shortcuts by introducing systematic interventions on
the training and test sides, including the boundary cases of `near-perfect` and
`worse than coin flip` (label flip). By using three different models, ranging
from classic to state-of-the-art, we demonstrate the presence of shortcut
learning in five simulated conditions. We analyze the results using a
regression model to understand how biases affect the class-conditional score
statistics.
