  Informally, a 'spurious correlation' is the dependence of a model on some
aspect of the input data that an analyst thinks shouldn't matter. In machine
learning, these have a know-it-when-you-see-it character; e.g., changing the
gender of a sentence's subject changes a sentiment predictor's output. To check
for spurious correlations, we can 'stress test' models by perturbing irrelevant
parts of input data and seeing if model predictions change. In this paper, we
study stress testing using the tools of causal inference. We introduce
counterfactual invariance as a formalization of the requirement that changing
irrelevant parts of the input shouldn't change model predictions. We connect
counterfactual invariance to out-of-domain model performance, and provide
practical schemes for learning (approximately) counterfactual invariant
predictors (without access to counterfactual examples). It turns out that both
the means and implications of counterfactual invariance depend fundamentally on
the true underlying causal structure of the data -- in particular, whether the
label causes the features or the features cause the label. Distinct causal
structures require distinct regularization schemes to induce counterfactual
invariance. Similarly, counterfactual invariance implies different domain shift
guarantees depending on the underlying causal structure. This theory is
supported by empirical results on text classification.
