  (This work has been submitted to the IEEE for possible publication. Copyright
may be transferred without notice, after which this version may no longer be
accessible.)
  To improve the efficiency of deep reinforcement learning (DRL)-based methods
for robot manipulator trajectory planning in random working environments, we
present three dense reward functions. These rewards differ from the traditional
sparse reward. First, a posture reward function is proposed to speed up the
learning process with a more reasonable trajectory by modeling the distance and
direction constraints, which can reduce the blindness of exploration. Second, a
stride reward function is proposed to improve the stability of the learning
process by modeling the distance and movement distance of joint constraints.
Finally, in order to further improve learning efficiency, we are inspired by
the cognitive process of human behavior and propose a stage incentive
mechanism, including a hard stage incentive reward function and a soft stage
incentive reward function. Extensive experiments show that the soft stage
incentive reward function is able to improve the convergence rate by up to
46.9% with the state-of-the-art DRL methods. The percentage increase in the
convergence mean reward was 4.4-15.5% and the percentage decreases with respect
to standard deviation were 21.9-63.2%. In the evaluation experiments, the
success rate of trajectory planning for a robot manipulator reached 99.6%.
