  Distance Metric Learning (DML) has attracted much attention in image
processing in recent years. This paper analyzes its impact on supervised
fine-tuning language models for Natural Language Processing (NLP)
classification tasks under few-shot learning settings. We investigated several
DML loss functions in training RoBERTa language models on known SentEval
Transfer Tasks datasets. We also analyzed the possibility of using proxy-based
DML losses during model inference.
  Our systematic experiments have shown that under few-shot learning settings,
particularly proxy-based DML losses can positively affect the fine-tuning and
inference of a supervised language model. Models tuned with a combination of
CCE (categorical cross-entropy loss) and ProxyAnchor Loss have, on average, the
best performance and outperform models with only CCE by about 3.27 percentage
points -- up to 10.38 percentage points depending on the training dataset.
