  Recent unsupervised contrastive representation learning follows a Single
Instance Multi-view (SIM) paradigm where positive pairs are usually constructed
with intra-image data augmentation. In this paper, we propose an effective
approach called Beyond Single Instance Multi-view (BSIM). Specifically, we
impose more accurate instance discrimination capability by measuring the joint
similarity between two randomly sampled instances and their mixture, namely
spurious-positive pairs. We believe that learning joint similarity helps to
improve the performance when encoded features are distributed more evenly in
the latent space. We apply it as an orthogonal improvement for unsupervised
contrastive representation learning, including current outstanding methods
SimCLR, MoCo, and BYOL. We evaluate our learned representations on many
downstream benchmarks like linear classification on ImageNet-1k and PASCAL VOC
2007, object detection on MS COCO 2017 and VOC, etc. We obtain substantial
gains with a large margin almost on all these tasks compared with prior arts.
