  Self-supervised learning (SSL) learns high-quality representations from large
pools of unlabeled training data. As datasets grow larger, it becomes crucial
to identify the examples that contribute the most to learning such
representations. This enables efficient SSL by reducing the volume of data
required. Nevertheless, quantifying the value of examples for SSL has remained
an open question. In this work, we address this problem for the first time, by
proving that examples that contribute the most to contrastive SSL are those
that have the most similar augmentations to other examples, in expectation. We
provide rigorous guarantees for the generalization performance of contrastive
learning on such subsets. Through extensive experiments, we show that we can
safely exclude 20% of examples from CIFAR100 and 40% from STL10 and
TinyImageNet, without affecting downstream task performance. In general,
subsets selected by our method outperform random subsets by over 3% across
these datasets. Interestingly, we also discover the subsets that contribute the
most to contrastive learning are those that contribute the least to supervised
learning. Code available at
https://github.com/bigml-cs-ucla/sas-data-efficient-contrastive-learning.
