  Existed pre-trained models have achieved state-of-the-art performance on
various text classification tasks. These models have proven to be useful in
learning universal language representations. However, the semantic discrepancy
between similar texts cannot be effectively distinguished by advanced
pre-trained models, which have a great influence on the performance of
hard-to-distinguish classes. To address this problem, we propose a novel
Contrastive Learning with Label Distance (CLLD) in this work. Inspired by
recent advances in contrastive learning, we specifically design a
classification method with label distance for learning contrastive classes.
CLLD ensures the flexibility within the subtle differences that lead to
different label assignments, and generates the distinct representations for
each class having similarity simultaneously. Extensive experiments on public
benchmarks and internal datasets demonstrate that our method improves the
performance of pre-trained models on classification tasks. Importantly, our
experiments suggest that the learned label distance relieve the adversarial
nature of interclasses.
