  Importance-weighted risk minimization is a key ingredient in many machine
learning algorithms for causal inference, domain adaptation, class imbalance,
and off-policy reinforcement learning. While the effect of importance weighting
is well-characterized for low-capacity misspecified models, little is known
about how it impacts over-parameterized, deep neural networks. This work is
inspired by recent theoretical results showing that on (linearly) separable
data, deep linear networks optimized by SGD learn weight-agnostic solutions,
prompting us to ask, for realistic deep networks, for which many practical
datasets are separable, what is the effect of importance weighting? We present
the surprising finding that while importance weighting impacts models early in
training, its effect diminishes over successive epochs. Moreover, while L2
regularization and batch normalization (but not dropout), restore some of the
impact of importance weighting, they express the effect via (seemingly) the
wrong abstraction: why should practitioners tweak the L2 regularization, and by
how much, to produce the correct weighting effect? Our experiments confirm
these findings across a range of architectures and datasets.
