  We develop data-driven reinforcement learning (RL) control designs for
input-affine nonlinear systems. We use Carleman linearization to express the
state-space representation of the nonlinear dynamical model in the Carleman
space, and develop a real-time algorithm that can learn nonlinear
state-feedback controllers using state and input measurements in the
infinite-dimensional Carleman space. Thereafter, we study the practicality of
having a finite-order truncation of the control signal, followed by its
closed-loop stability analysis. Finally, we develop two additional designs that
can learn structured as well as sparse representations of the RL-based
nonlinear controller, and provide theoretical conditions for ensuring their
closed-loop stability. We present numerical examples to show how our proposed
method generates closed-loop responses that are close to the optimal
performance of the nonlinear plant. We also compare our designs to other
data-driven nonlinear RL control methods such as those based on neural
networks, and illustrate their relative advantages and drawbacks.
