  Time-optimal path tracking, as a significant tool for industrial robots, has
attracted the attention of numerous researchers. In most time-optimal path
tracking problems, the actuator torque constraints are assumed to be
conservative, which ignores the motor characteristic; i.e., the actuator torque
constraints are velocity-dependent, and the relationship between torque and
velocity is piecewise linear. However, considering that the motor
characteristics increase the solving difficulty, in this study, an improved
Q-learning algorithm for robotic time-optimal path tracking using prior
knowledge is proposed. After considering the limitations of the Q-learning
algorithm, an improved action-value function is proposed to improve the
convergence rate. The proposed algorithms use the idea of reward and penalty,
rewarding the actions that satisfy constraint conditions and penalizing the
actions that break constraint conditions, to finally obtain a time-optimal
trajectory that satisfies the constraint conditions. The effectiveness of the
algorithms is verified by experiments.
