  A fundamental challenge in artificial intelligence is learning useful
representations of data that yield good performance on a downstream task,
without overfitting to spurious input features. Extracting such task-relevant
predictive information is particularly difficult for real-world datasets. In
this work, we propose Contrastive Input Morphing (CIM), a representation
learning framework that learns input-space transformations of the data to
mitigate the effect of irrelevant input features on downstream performance. Our
method leverages a perceptual similarity metric via a triplet loss to ensure
that the transformation preserves task-relevant information.Empirically, we
demonstrate the efficacy of our approach on tasks which typically suffer from
the presence of spurious correlations: classification with nuisance
information, out-of-distribution generalization, and preservation of subgroup
accuracies. We additionally show that CIM is complementary to other mutual
information-based representation learning techniques, and demonstrate that it
improves the performance of variational information bottleneck (VIB) when used
together.
