  A central goal of machine learning is to learn robust representations that
capture the causal relationship between inputs features and output labels.
However, minimizing empirical risk over finite or biased datasets often results
in models latching on to spurious correlations between the training
input/output pairs that are not fundamental to the problem at hand. In this
paper, we define and analyze robust and spurious representations using the
information-theoretic concept of minimal sufficient statistics. We prove that
even when there is only bias of the input distribution (i.e. covariate shift),
models can still pick up spurious features from their training data. Group
distributionally robust optimization (DRO) provides an effective tool to
alleviate covariate shift by minimizing the worst-case training loss over a set
of pre-defined groups. Inspired by our analysis, we demonstrate that group DRO
can fail when groups do not directly account for various spurious correlations
that occur in the data. To address this, we further propose to minimize the
worst-case losses over a more flexible set of distributions that are defined on
the joint distribution of groups and instances, instead of treating each group
as a whole at optimization time. Through extensive experiments on one image and
two language tasks, we show that our model is significantly more robust than
comparable baselines under various partitions. Our code is available at
https://github.com/violet-zct/group-conditional-DRO.
