  We propose a computationally efficient approach to safe reinforcement
learning (RL) for frequency regulation in power systems with high levels of
variable renewable energy resources. The approach draws on set-theoretic
control techniques to craft a neural network-based control policy that is
guaranteed to satisfy safety-critical state constraints, without needing to
solve a model predictive control or projection problem in real time. By
exploiting the properties of robust controlled-invariant polytopes, we
construct a novel, closed-form "safety-filter" that enables end-to-end safe
learning using any policy gradient-based RL algorithm. We then apply the safety
filter in conjunction with the deep deterministic policy gradient (DDPG)
algorithm to regulate frequency in a modified 9-bus power system, and show that
the learned policy is more cost-effective than robust linear feedback control
techniques while maintaining the same safety guarantee. We also show that the
proposed paradigm outperforms DDPG augmented with constraint violation
penalties.
