  As more inverter-connected renewable resources are integrated into the grid,
frequency stability may degrade because of the reduction in mechanical inertia
and damping. A common approach to mitigate this degradation in performance is
to use the power electronic interfaces of the renewable resources for primary
frequency control. Since inverter-connected resources can realize almost
arbitrary responses to frequency changes, they are not limited to reproducing
the linear droop behaviors. To fully leverage their capabilities, reinforcement
learning (RL) has emerged as a popular method to design nonlinear controllers
to optimize a host of objective functions.
  Because both inverter-connected resources and synchronous generators would be
a significant part of the grid in the near and intermediate future, the learned
controller of the former should be stabilizing with respect to the nonlinear
dynamics of the latter. To overcome this challenge, we explicitly engineer the
structure of neural network-based controllers such that they guarantee system
stability by construction, through the use of a Lyapunov function. A recurrent
neural network architecture is used to efficiently train the controllers. The
resulting controllers only use local information and outperform optimal linear
droop as well as other state-of-the-art learning approaches.
