  In recent times, a variety of Reinforcement Learning (RL) algorithms have
been proposed for optimal tracking problem of continuous time nonlinear systems
with input constraints. Most of these algorithms are based on the notion of
uniform ultimate boundedness (UUB) stability, in which normally higher learning
rates are avoided in order to restrict oscillations in state error to smaller
values. However, this comes at the cost of higher convergence time of critic
neural network weights. This paper addresses that problem by proposing a novel
tuning law containing a variable gain gradient descent for critic neural
network that can adjust the learning rate based on Hamilton-Jacobi-Bellman
(HJB) error. By allowing high learning rate the proposed variable gain gradient
descent tuning law could improve the convergence time of critic neural network
weights. Simultaneously, it also results in tighter residual set, on which
trajectories of augmented system converge to, leading to smaller oscillations
in state error. A tighter bound for UUB stability of the proposed update
mechanism is proved. Numerical studies are then presented to validate the
robust Reinforcement Learning control scheme in controlling a continuous time
nonlinear system.
