  Online reinforcement learning is concerned with training an agent on-the-fly
via dynamic interaction with the environment. Here, due to the specifics of the
application, it is not generally possible to perform long pre-training, as it
is commonly done in off-line, model-free approaches, which are akin to dynamic
programming. Such applications may be found more frequently in industry, rather
than in pure digital fields, such as cloud services, video games, database
management, etc., where reinforcement learning has been demonstrating success.
Online reinforcement learning, in contrast, is more akin to classical control,
which utilizes some model knowledge about the environment. Stability of the
closed-loop (agent plus the environment) is a major challenge for such online
approaches. In this paper, we tackle this problem by a special fusion of online
reinforcement learning with elements of classical control, namely, based on the
Lyapunov theory of stability. The idea is to start the agent at once, without
pre-training, and learn approximately optimal policy under specially designed
constraints, which guarantee stability. The resulting approach was tested in an
extensive experimental study with a mobile robot. A nominal parking controller
was used as a baseline. It was observed that the suggested agent could always
successfully park the robot, while significantly improving the cost. While many
approaches may be exploited for mobile robot control, we suggest that the
experiments showed the promising potential of online reinforcement learning
agents based on Lyapunov-like constraints. The presented methodology may be
utilized in safety-critical, industrial applications where stability is
necessary.
