  Contrastive Self-supervised Learning (CSL) is a practical solution that
learns meaningful visual representations from massive data in an unsupervised
approach. The ordinary CSL embeds the features extracted from neural networks
onto specific topological structures. During the training progress, the
contrastive loss draws the different views of the same input together while
pushing the embeddings from different inputs apart. One of the drawbacks of CSL
is that the loss term requires a large number of negative samples to provide
better mutual information bound ideally. However, increasing the number of
negative samples by larger running batch size also enhances the effects of
false negatives: semantically similar samples are pushed apart from the anchor,
hence downgrading downstream performance. In this paper, we tackle this problem
by introducing a simple but effective contrastive learning framework. The key
insight is to employ siamese-style metric loss to match intra-prototype
features, while increasing the distance between inter-prototype features. We
conduct extensive experiments on various benchmarks where the results
demonstrate the effectiveness of our method on improving the quality of visual
representations. Specifically, our unsupervised pre-trained ResNet-50 with a
linear probe, out-performs the fully-supervised trained version on the
ImageNet-1K dataset.
