  Contrastive learning has achieved great success in self-supervised visual
representation learning, but existing approaches mostly ignored spatial
information which is often crucial for visual representation. This paper
presents heterogeneous contrastive learning (HCL), an effective approach that
adds spatial information to the encoding stage to alleviate the learning
inconsistency between the contrastive objective and strong data augmentation
operations. We demonstrate the effectiveness of HCL by showing that (i) it
achieves higher accuracy in instance discrimination and (ii) it surpasses
existing pre-training methods in a series of downstream tasks while shrinking
the pre-training costs by half. More importantly, we show that our approach
achieves higher efficiency in visual representations, and thus delivers a key
message to inspire the future research of self-supervised visual representation
learning.
