  To solve the coupling problem of control loops and the adaptive parameter
tuning problem in the multi-input multi-output (MIMO) PID control system, a
self-adaptive LSAC-PID algorithm is proposed based on deep reinforcement
learning (RL) and Lyapunov-based reward shaping in this paper. For complex and
unknown mobile robot control environment, an RL-based MIMO PID hybrid control
strategy is firstly presented. According to the dynamic information and
environmental feedback of the mobile robot, the RL agent can output the optimal
MIMO PID parameters in real time, without knowing mathematical model and
decoupling multiple control loops. Then, to improve the convergence speed of RL
and the stability of mobile robots, a Lyapunov-based reward shaping soft
actor-critic (LSAC) algorithm is proposed based on Lyapunov theory and
potential-based reward shaping method. The convergence and optimality of the
algorithm are proved in terms of the policy evaluation and improvement step of
soft policy iteration. In addition, for line-following robots, the region
growing method is improved to adapt to the influence of forks and environmental
interference. Through comparison, test and cross-validation, the simulation and
real-environment experimental results all show good performance of the proposed
LSAC-PID tuning algorithm.
