  Machine learning models often perform poorly on subgroups that are
underrepresented in the training data. Yet, little is understood on the
variation in mechanisms that cause subpopulation shifts, and how algorithms
generalize across such diverse shifts at scale. In this work, we provide a
fine-grained analysis of subpopulation shift. We first propose a unified
framework that dissects and explains common shifts in subgroups. We then
establish a comprehensive benchmark of 20 state-of-the-art algorithms evaluated
on 12 real-world datasets in vision, language, and healthcare domains. With
results obtained from training over 10,000 models, we reveal intriguing
observations for future progress in this space. First, existing algorithms only
improve subgroup robustness over certain types of shifts but not others.
Moreover, while current algorithms rely on group-annotated validation data for
model selection, we find that a simple selection criterion based on worst-class
accuracy is surprisingly effective even without any group information. Finally,
unlike existing works that solely aim to improve worst-group accuracy (WGA), we
demonstrate the fundamental tradeoff between WGA and other important metrics,
highlighting the need to carefully choose testing metrics. Code and data are
available at: https://github.com/YyzHarry/SubpopBench.
