  Deep learning models have been shown to learn spurious correlations from data
that sometimes lead to systematic failures for certain subpopulations. Prior
work has typically diagnosed this by crowdsourcing annotations for various
protected attributes and measuring performance, which is both expensive to
acquire and difficult to scale. In this work, we propose UDIS, an unsupervised
algorithm for surfacing and analyzing such failure modes. UDIS identifies
subpopulations via hierarchical clustering of dataset embeddings and surfaces
systematic failure modes by visualizing low performing clusters along with
their gradient-weighted class-activation maps. We show the effectiveness of
UDIS in identifying failure modes in models trained for image classification on
the CelebA and MSCOCO datasets.
