  Self-supervised representation learning has made significant leaps fueled by
progress in contrastive learning, which seeks to learn transformations that
embed positive input pairs nearby, while pushing negative pairs far apart.
While positive pairs can be generated reliably (e.g., as different views of the
same image), it is difficult to accurately establish negative pairs, defined as
samples from different images regardless of their semantic content or visual
features. A fundamental problem in contrastive learning is mitigating the
effects of false negatives. Contrasting false negatives induces two critical
issues in representation learning: discarding semantic information and slow
convergence. In this paper, we propose novel approaches to identify false
negatives, as well as two strategies to mitigate their effect, i.e. false
negative elimination and attraction, while systematically performing rigorous
evaluations to study this problem in detail. Our method exhibits consistent
improvements over existing contrastive learning-based methods. Without labels,
we identify false negatives with 40% accuracy among 1000 semantic classes on
ImageNet, and achieve 5.8% absolute improvement in top-1 accuracy over the
previous state-of-the-art when finetuning with 1% labels. Our code is available
at https://github.com/google-research/fnc.
