  We propose a piecewise learning framework for controlling nonlinear systems
with unknown dynamics. While model-based reinforcement learning techniques in
terms of some basis functions are well known in the literature, when it comes
to more complex dynamics, only a local approximation of the model can be
obtained using a limited number of bases. The complexity of the identifier and
the controller can be considerably high if obtaining an approximation over a
larger domain is desired. To overcome this limitation, we propose a general
piecewise nonlinear framework where each piece is responsible for locally
learning and controlling over some region of the domain. We obtain rigorous
uncertainty bounds for the learned piecewise models. The piecewise affine (PWA)
model is then studied as a special case, for which we propose an
optimization-based verification technique for stability analysis of the
closed-loop system. Accordingly, given a time-discretization of the learned
{PWA} system, we iteratively search for a common piecewise Lyapunov function in
a set of positive definite functions, where a non-monotonic convergence is
allowed. This Lyapunov candidate is verified on the uncertain system to either
provide a certificate for stability or find a counter-example when it fails.
This counter-example is added to a set of samples to facilitate the further
learning of a Lyapunov function. We demonstrate the results on two examples and
show that the proposed approach yields a less conservative region of attraction
(ROA) compared with alternative state-of-the-art approaches. Moreover, we
provide the runtime results to demonstrate potentials of the proposed framework
in real-world implementations.
