  Recent strides in large language models (LLMs) have yielded remarkable
performance, leveraging reinforcement learning from human feedback (RLHF) to
significantly enhance generation and alignment capabilities. However, RLHF
encounters numerous challenges, including the objective mismatch issue, leading
to suboptimal performance in Natural Language Understanding (NLU) tasks. To
address this limitation, we propose a novel Reinforcement Learning framework
enhanced with Label-sensitive Reward (RLLR) to amplify the performance of LLMs
in NLU tasks. By incorporating label-sensitive pairs into reinforcement
learning, our method aims to adeptly capture nuanced label-sensitive semantic
features during RL, thereby enhancing natural language understanding.
Experiments conducted on five diverse foundation models across eight tasks
showcase promising results. In comparison to Supervised Fine-tuning models
(SFT), RLLR demonstrates an average performance improvement of 1.54%. Compared
with RLHF models, the improvement averages at 0.69%. These results reveal the
effectiveness of our method for LLMs in NLU tasks. Code and data available at:
https://github.com/MagiaSN/ACL2024_RLLR.
