  Contrastive learning (CL) has achieved astonishing progress in computer
vision, speech, and natural language processing fields recently with
self-supervised learning. However, CL approach to the supervised setting is not
fully explored, especially for the natural language understanding
classification task. Intuitively, the class label itself has the intrinsic
ability to perform hard positive/negative mining, which is crucial for CL.
Motivated by this, we propose a novel label anchored contrastive learning
approach (denoted as LaCon) for language understanding. Specifically, three
contrastive objectives are devised, including a multi-head instance-centered
contrastive loss (ICL), a label-centered contrastive loss (LCL), and a label
embedding regularizer (LER). Our approach does not require any specialized
network architecture or any extra data augmentation, thus it can be easily
plugged into existing powerful pre-trained language models. Compared to the
state-of-the-art baselines, LaCon obtains up to 4.1% improvement on the popular
datasets of GLUE and CLUE benchmarks. Besides, LaCon also demonstrates
significant advantages under the few-shot and data imbalance settings, which
obtains up to 9.4% improvement on the FewGLUE and FewCLUE benchmarking tasks.
