  Contrastive learning (CL) continuously achieves significant breakthroughs
across multiple domains. However, the most common InfoNCE-based methods suffer
from some dilemmas, such as \textit{uniformity-tolerance dilemma} (UTD) and
\textit{gradient reduction}, both of which are related to a $\mathcal{P}_{ij}$
term. It has been identified that UTD can lead to unexpected performance
degradation. We argue that the fixity of temperature is to blame for UTD. To
tackle this challenge, we enrich the CL loss family by presenting a Model-Aware
Contrastive Learning (MACL) strategy, whose temperature is adaptive to the
magnitude of alignment that reflects the basic confidence of the instance
discrimination task, then enables CL loss to adjust the penalty strength for
hard negatives adaptively. Regarding another dilemma, the gradient reduction
issue, we derive the limits of an involved gradient scaling factor, which
allows us to explain from a unified perspective why some recent approaches are
effective with fewer negative samples, and summarily present a gradient
reweighting to escape this dilemma. Extensive remarkable empirical results in
vision, sentence, and graph modality validate our approach's general
improvement for representation learning and downstream tasks.
