  Deploying large language models (LLMs) poses challenges in terms of resource
limitations and inference efficiency. To address these challenges, recent
research has focused on using smaller task-specific language models, which are
enhanced by distilling the knowledge rationales generated by LLMs. However,
previous works mostly emphasize the effectiveness of positive knowledge, while
overlooking the knowledge noise and the exploration of negative knowledge. In
this paper, we first propose a general approach called quality-guided
contrastive rationale distillation for reasoning capacity learning, considering
contrastive learning perspectives. For the learning of positive knowledge, we
collect positive rationales through self-consistency to denoise the LLM
rationales generated by temperature sampling. For the negative knowledge
distillation, we generate negative rationales using temperature sampling for
the iteration-before smaller language models themselves. Finally, a contrastive
loss is designed to better distill the positive and negative rationales into
the smaller language model, where an online-update discriminator is used to
judge the qualities of rationales and assign weights for better optimizing the
training process. Through extensive experiments on multiple reasoning tasks, we
demonstrate that our method consistently outperforms the previous distillation
methods and produces higher-quality rationales.
