  Although Deep Reinforcement Learning (DRL) has achieved notable success in
numerous robotic applications, designing a high-performing reward function
remains a challenging task that often requires substantial manual input.
Recently, Large Language Models (LLMs) have been extensively adopted to address
tasks demanding in-depth common-sense knowledge, such as reasoning and
planning. Recognizing that reward function design is also inherently linked to
such knowledge, LLM offers a promising potential in this context. Motivated by
this, we propose in this work a novel LLM framework with a self-refinement
mechanism for automated reward function design. The framework commences with
the LLM formulating an initial reward function based on natural language
inputs. Then, the performance of the reward function is assessed, and the
results are presented back to the LLM for guiding its self-refinement process.
We examine the performance of our proposed framework through a variety of
continuous robotic control tasks across three diverse robotic systems. The
results indicate that our LLM-designed reward functions are able to rival or
even surpass manually designed reward functions, highlighting the efficacy and
applicability of our approach.
