  This paper considers the problem of real-time control and learning in dynamic
systems subjected to parametric uncertainties. We propose a combination of a
Reinforcement Learning (RL) based policy in the outer loop suitably chosen to
ensure stability and optimality for the nominal dynamics, together with
Adaptive Control (AC) in the inner loop so that in real-time AC contracts the
closed-loop dynamics towards a stable trajectory traced out by RL. Two classes
of nonlinear dynamic systems are considered, both of which are control-affine.
The first class of dynamic systems utilizes equilibrium points %with expansion
forms around these points and a Lyapunov approach while second class of
nonlinear systems uses contraction theory. AC-RL controllers are proposed for
both classes of systems and shown to lead to online policies that guarantee
stability using a high-order tuner and accommodate parametric uncertainties and
magnitude limits on the input. In addition to establishing a stability
guarantee with real-time control, the AC-RL controller is also shown to lead to
parameter learning with persistent excitation for the first class of systems.
Numerical validations of all algorithms are carried out using a quadrotor
landing task on a moving platform.
