  Distributionally robust optimization (DRO) and invariant risk minimization
(IRM) are two popular methods proposed to improve out-of-distribution (OOD)
generalization performance of machine learning models. While effective for
small models, it has been observed that these methods can be vulnerable to
overfitting with large overparameterized models. This work proposes a
principled method, \textbf{M}odel \textbf{A}gnostic sam\textbf{PL}e
r\textbf{E}weighting (\textbf{MAPLE}), to effectively address OOD problem,
especially in overparameterized scenarios. Our key idea is to find an effective
reweighting of the training samples so that the standard empirical risk
minimization training of a large model on the weighted training data leads to
superior OOD generalization performance. The overfitting issue is addressed by
considering a bilevel formulation to search for the sample reweighting, in
which the generalization complexity depends on the search space of sample
weights instead of the model size. We present theoretical analysis in linear
case to prove the insensitivity of MAPLE to model size, and empirically verify
its superiority in surpassing state-of-the-art methods by a large margin. Code
is available at \url{https://github.com/x-zho14/MAPLE}.
