  Reinforcement learning (RL) in the context of control systems offers wide
possibilities of controller adaptation. Given an infinite-horizon cost
function, the so-called critic of RL approximates it with a neural net and
sends this information to the controller (called "actor"). However, the issue
of closed-loop stability under an RL-method is still not fully addressed. Since
the critic delivers merely an approximation to the value function of the
corresponding infinite-horizon problem, no guarantee can be given in general as
to whether the actor's actions stabilize the system. Different approaches to
this issue exist. The current work offers a particular one, which, starting
with a (not necessarily smooth) control Lyapunov function (CLF), derives an
online RL-scheme in such a way that practical semi-global stability property of
the closed-loop can be established. The approach logically continues the work
of the authors on parameterized controllers and Lyapunov-like constraints for
RL, whereas the CLF now appears merely in one of the constraints of the control
scheme. The analysis of the closed-loop behavior is done in a sample-and-hold
(SH) manner thus offering a certain insight into the digital realization. The
case study with a non-holonomic integrator shows the capabilities of the
derived method to optimize the given cost function compared to a nominal
stabilizing controller.
