  Despite the rapid development of machine learning algorithms for domain
generalization (DG), there is no clear empirical evidence that the existing DG
algorithms outperform the classic empirical risk minimization (ERM) across
standard benchmarks. To better understand this phenomenon, we investigate
whether there are benefits of DG algorithms over ERM through the lens of label
noise. Specifically, our finite-sample analysis reveals that label noise
exacerbates the effect of spurious correlations for ERM, undermining
generalization. Conversely, we illustrate that DG algorithms exhibit implicit
label-noise robustness during finite-sample training even when spurious
correlation is present. Such desirable property helps mitigate spurious
correlations and improve generalization in synthetic experiments. However,
additional comprehensive experiments on real-world benchmark datasets indicate
that label-noise robustness does not necessarily translate to better
performance compared to ERM. We conjecture that the failure mode of ERM arising
from spurious correlations may be less pronounced in practice.
