  The goal of program synthesis, or code generation, is to generate executable
code based on given descriptions. Recently, there has been an increasing number
of studies employing reinforcement learning (RL) to improve the performance of
large language models (LLMs) for code. However, current representative works
either rely solely on offline frameworks, limiting the exploration of new
sample spaces, or fall short in the utilization of unit test signals, not
accounting for specific error locations within the code. To address these
issues, we propose RLTF, i.e., Reinforcement Learning from Unit Test Feedback,
a novel online RL framework with unit test feedback of multi-granularity for
refining code LLMs. Our approach generates data in real-time during training
and simultaneously utilizes fine-grained feedback signals to guide the model
towards producing higher-quality code. Extensive experiments show that RLTF
achieves state-of-the-art performance on the APPS and the MBPP benchmarks. Our
code is available at: https://github.com/Zyq-scut/RLTF.
