  Contrastive learning, along with its variations, has been a highly effective
self-supervised learning method across diverse domains. Contrastive learning
measures the distance between representations using cosine similarity and uses
cross-entropy for representation learning. Within the same framework of
cosine-similarity-based representation learning, margins have played a
significant role in enhancing face and speaker recognition tasks.
Interestingly, despite the shared reliance on the same similarity metrics and
objective functions, contrastive learning has not actively adopted margins.
Furthermore, decision-boundary-based explanations are the only ones that have
been used to explain the effect of margins in contrastive learning. In this
work, we propose a new perspective to understand the role of margins based on
gradient analysis. Based on the new perspective, we analyze how margins affect
gradients of contrastive learning and separate the effect into more elemental
levels. We separately analyze each and provide possible directions for
improving contrastive learning. Our experimental results demonstrate that
emphasizing positive samples and scaling gradients depending on positive sample
angles and logits are the keys to improving the generalization performance of
contrastive learning in both seen and unseen datasets, and other factors can
only marginally improve performance.
