  Group robustness strategies aim to mitigate learned biases in deep learning
models that arise from spurious correlations present in their training
datasets. However, most existing methods rely on the access to the label
distribution of the groups, which is time-consuming and expensive to obtain. As
a result, unsupervised group robustness strategies are sought. Based on the
insight that a trained model's classification strategies can be inferred
accurately based on explainability heatmaps, we introduce ExMap, an
unsupervised two stage mechanism designed to enhance group robustness in
traditional classifiers. ExMap utilizes a clustering module to infer
pseudo-labels based on a model's explainability heatmaps, which are then used
during training in lieu of actual labels. Our empirical studies validate the
efficacy of ExMap - We demonstrate that it bridges the performance gap with its
supervised counterparts and outperforms existing partially supervised and
unsupervised methods. Additionally, ExMap can be seamlessly integrated with
existing group robustness learning strategies. Finally, we demonstrate its
potential in tackling the emerging issue of multiple shortcut
mitigation\footnote{Code available at \url{https://github.com/rwchakra/exmap}}.
