  This paper addresses reinforcement learning based, direct signal tracking
control with an objective of developing mathematically suitable and practically
useful design approaches. Specifically, we aim to provide reliable and easy to
implement designs in order to reach reproducible neural network-based
solutions. Our proposed new design takes advantage of two control design
frameworks: a reinforcement learning based, data-driven approach to provide the
needed adaptation and (sub)optimality, and a backstepping based approach to
provide closed-loop system stability framework. We develop this work based on
an established direct heuristic dynamic programming (dHDP) learning paradigm to
perform online learning and adaptation and a backstepping design for a class of
important nonlinear dynamics described as Euler-Lagrange systems. We provide a
theoretical guarantee for the stability of the overall dynamic system, weight
convergence of the approximating nonlinear neural networks, and the Bellman
(sub)optimality of the resulted control policy. We use simulations to
demonstrate significantly improved design performance of the proposed approach
over the original dHDP.
