  Recent progress in contrastive learning has revolutionized unsupervised
representation learning. Concretely, multiple views (augmentations) from the
same image are encouraged to map to the similar embeddings, while views from
different images are pulled apart. In this paper, through visualizing and
diagnosing classification errors, we observe that current contrastive models
are ineffective at localizing the foreground object, limiting their ability to
extract discriminative high-level features. This is due to the fact that view
generation process considers pixels in an image uniformly. To address this
problem, we propose a data-driven approach for learning invariance to
backgrounds. It first estimates foreground saliency in images and then creates
augmentations by copy-and-pasting the foreground onto a variety of backgrounds.
The learning still follows the instance discrimination pretext task, so that
the representation is trained to disregard background content and focus on the
foreground. We study a variety of saliency estimation methods, and find that
most methods lead to improvements for contrastive learning. With this approach
(DiLo), significant performance is achieved for self-supervised learning on
ImageNet classification, and also for object detection on PASCAL VOC and
MSCOCO.
