  Batch normalization (BN) is a ubiquitous technique for training deep neural
networks that accelerates their convergence to reach higher accuracy. However,
we demonstrate that BN comes with a fundamental drawback: it incentivizes the
model to rely on low-variance features that are highly specific to the training
(in-domain) data, hurting generalization performance on out-of-domain examples.
In this work, we investigate this phenomenon by first showing that removing BN
layers across a wide range of architectures leads to lower out-of-domain and
corruption errors at the cost of higher in-domain errors. We then propose
Counterbalancing Teacher (CT), a method which leverages a frozen copy of the
same model without BN as a teacher to enforce the student network's learning of
robust representations by substantially adapting its weights through a
consistency loss function. This regularization signal helps CT perform well in
unforeseen data shifts, even without information from the target domain as in
prior works. We theoretically show in an overparameterized linear regression
setting why normalization leads to a model's reliance on such in-domain
features, and empirically demonstrate the efficacy of CT by outperforming
several baselines on robustness benchmarks such as CIFAR-10-C, CIFAR-100-C, and
VLCS.
