  Recently, generalization on out-of-distribution (OOD) data with correlation
shift has attracted great attentions. The correlation shift is caused by the
spurious attributes that correlate to the class label, as the correlation
between them may vary in training and test data. For such a problem, we show
that given the class label, the models that are conditionally independent of
spurious attributes are OOD generalizable. Based on this, a metric Conditional
Spurious Variation (CSV) which controls the OOD generalization error, is
proposed to measure such conditional independence. To improve the OOD
generalization, we regularize the training process with the proposed CSV. Under
mild assumptions, our training objective can be formulated as a
nonconvex-concave mini-max problem. An algorithm with a provable convergence
rate is proposed to solve the problem. Extensive empirical results verify our
algorithm's efficacy in improving OOD generalization.
