  We investigate the important problem of certifying stability of reinforcement
learning policies when interconnected with nonlinear dynamical systems. We show
that by regulating the input-output gradients of policies, strong guarantees of
robust stability can be obtained based on a proposed semidefinite programming
feasibility problem. The method is able to certify a large set of stabilizing
controllers by exploiting problem-specific structures; furthermore, we analyze
and establish its (non)conservatism. Empirical evaluations on two decentralized
control tasks, namely multi-flight formation and power system frequency
regulation, demonstrate that the reinforcement learning agents can have high
performance within the stability-certified parameter space, and also exhibit
stable learning behaviors in the long run.
