  We present a technique for learning control Lyapunov-like functions, which
are used in turn to synthesize controllers for nonlinear dynamical systems that
can stabilize the system, or satisfy specifications such as remaining inside a
safe set, or eventually reaching a target set while remaining inside a safe
set. The learning framework uses a demonstrator that implements a black-box,
untrusted strategy presumed to solve the problem of interest, a learner that
poses finitely many queries to the demonstrator to infer a candidate function,
and a verifier that checks whether the current candidate is a valid control
Lyapunov function. The overall learning framework is iterative, eliminating a
set of candidates on each iteration using the counterexamples discovered by the
verifier and the demonstrations over these counterexamples. We prove its
convergence using ellipsoidal approximation techniques from convex
optimization. We also implement this scheme using nonlinear MPC controllers to
serve as demonstrators for a set of state and trajectory stabilization problems
for nonlinear dynamical systems. We show how the verifier can be constructed
efficiently using convex relaxations of the verification problem for polynomial
systems to semi-definite programming (SDP) problem instances. Our approach is
able to synthesize relatively simple polynomial control Lyapunov functions, and
in that process replace the MPC using a guaranteed and computationally less
expensive controller.
