  In deterministic systems, reinforcement learning-based online approximate
optimal control methods typically require a restrictive persistence of
excitation (PE) condition for convergence. This paper presents a concurrent
learning-based solution to the online approximate optimal regulation problem
that eliminates the need for PE. The development is based on the observation
that given a model of the system, the Bellman error, which quantifies the
deviation of the system Hamiltonian from the optimal Hamiltonian, can be
evaluated at any point in the state space. Further, a concurrent learning-based
parameter identifier is developed to compensate for parametric uncertainty in
the plant dynamics. Uniformly ultimately bounded (UUB) convergence of the
system states to the origin, and UUB convergence of the developed policy to the
optimal policy are established using a Lyapunov-based analysis, and simulations
are performed to demonstrate the performance of the developed controller.
