  Recent language models have shown remarkable performance on natural language
understanding (NLU) tasks. However, they are often sub-optimal when faced with
ambiguous samples that can be interpreted in multiple ways, over-confidently
predicting a single label without consideration for its correctness. To address
this issue, we propose a novel self-knowledge distillation method that enables
models to learn label distributions more accurately by leveraging knowledge
distilled from their lower layers. This approach also includes a learning phase
that re-calibrates the unnecessarily strengthened confidence for training
samples judged as extremely ambiguous based on the distilled distribution
knowledge. We validate our method on diverse NLU benchmark datasets and the
experimental results demonstrate its effectiveness in producing better label
distributions. Particularly, through the process of re-calibrating the
confidence for highly ambiguous samples, the issue of over-confidence when
predictions for unseen samples do not match with their ground-truth labels has
been significantly alleviated. This has been shown to contribute to generating
better distributions than the existing state-of-the-art method. Moreover, our
method is more efficient in training the models compared to the existing
method, as it does not involve additional training processes to refine label
distributions.
