  While neural networks have shown remarkable success on classification tasks
in terms of average-case performance, they often fail to perform well on
certain groups of the data. Such group information may be expensive to obtain;
thus, recent works in robustness and fairness have proposed ways to improve
worst-group performance even when group labels are unavailable for the training
data. However, these methods generally underperform methods that utilize group
information at training time. In this work, we assume access to a small number
of group labels alongside a larger dataset without group labels. We propose
BARACK, a simple two-step framework to utilize this partial group information
to improve worst-group performance: train a model to predict the missing group
labels for the training data, and then use these predicted group labels in a
robust optimization objective. Theoretically, we provide generalization bounds
for our approach in terms of the worst-group performance, which scale with
respect to both the total number of training points and the number of training
points with group labels. Empirically, our method outperforms the baselines
that do not use group information, even when only 1-33% of points have group
labels. We provide ablation studies to support the robustness and extensibility
of our framework.
