  Stable concurrent learning and control of dynamical systems is the subject of
adaptive control. Despite being an established field with many practical
applications and a rich theory, much of the development in adaptive control for
nonlinear systems revolves around a few key algorithms. By exploiting strong
connections between classical adaptive nonlinear control techniques and recent
progress in optimization and machine learning, we show that there exists
considerable untapped potential in algorithm development for both adaptive
nonlinear control and adaptive dynamics prediction. We begin by introducing
first-order adaptation laws inspired by natural gradient descent and mirror
descent. We prove that when there are multiple dynamics consistent with the
data, these non-Euclidean adaptation laws implicitly regularize the learned
model. Local geometry imposed during learning thus may be used to select
parameter vectors -- out of the many that will achieve perfect tracking or
prediction -- for desired properties such as sparsity. We apply this result to
regularized dynamics predictor and observer design, and as concrete examples,
we consider Hamiltonian systems, Lagrangian systems, and recurrent neural
networks. We subsequently develop a variational formalism based on the Bregman
Lagrangian. We show that its Euler Lagrange equations lead to natural gradient
and mirror descent-like adaptation laws with momentum, and we recover their
first-order analogues in the infinite friction limit. We illustrate our
analyses with simulations demonstrating our theoretical results.
