  Recently, self-supervised learning has attracted great attention, since it
only requires unlabeled data for model training. Contrastive learning is one
popular method for self-supervised learning and has achieved promising
empirical performance. However, the theoretical understanding of its
generalization ability is still limited. To this end, we define a kind of
$(\sigma,\delta)$-measure to mathematically quantify the data augmentation, and
then provide an upper bound of the downstream classification error rate based
on the measure. It reveals that the generalization ability of contrastive
self-supervised learning is related to three key factors: alignment of positive
samples, divergence of class centers, and concentration of augmented data. The
first two factors are properties of learned representations, while the third
one is determined by pre-defined data augmentation. We further investigate two
canonical contrastive losses, InfoNCE and cross-correlation, to show how they
provably achieve the first two factors. Moreover, we conduct experiments to
study the third factor, and observe a strong correlation between downstream
performance and the concentration of augmented data.
