  In reinforcement learning, reward shaping is an efficient way to guide the
learning process of an agent, as the reward can indicate the optimal policy of
the task. The potential-based reward shaping framework was proposed to
guarantee policy invariance after reward shaping, where a potential function is
used to calculate the shaping reward. In former work, we proposed a novel
adaptive potential function (APF) method to learn the potential function
concurrently with training the agent based on information collected by the
agent during the training process, and examined the APF method in discrete
action space scenarios. This paper investigates the feasibility of using APF in
solving continuous-reaching tasks in a real-world robotic scenario with
continuous action space. We combine the Deep Deterministic Policy Gradient
(DDPG) algorithm and our proposed method to form a new algorithm called
APF-DDPG. To compare APF-DDPG with DDPG, we designed a task where the agent
learns to control Baxter's right arm to reach a goal position. The experimental
results show that the APF-DDPG algorithm outperforms the DDPG algorithm on both
learning speed and robustness.
