  The vanishing inertia of synchronous generators in transmission systems
requires the utilization of renewables for inertial support. These are often
connected to the distribution system and their support should be coordinated to
avoid violation of grid limits. To this end, this paper presents the
Physics-informed Actor-Critic (PI-AC) algorithm for coordination of Virtual
Inertia (VI) from renewable Inverter-based Resources (IBRs) in power
distribution systems. Acquiring a model of the distribution grid can be
difficult, since certain parts are often unknown or the parameters are highly
uncertain. To favor model-free coordination, Reinforcement Learning (RL)
methods can be employed, necessitating a substantial level of training
beforehand. The PI-AC is a RL algorithm that integrates the physical behavior
of the power system into the Actor-Critic (AC) approach in order to achieve
faster learning. To this end, we regularize the loss function with an
aggregated power system dynamics model based on the swing equation. Throughout
this paper, we explore the PI-AC functionality in a case study with the CIGRE
14-bus and IEEE 37-bus power distribution system in various grid settings. The
PI-AC is able to achieve better rewards and faster learning than the
exclusively data-driven AC algorithm and the metaheuristic Genetic Algorithm
(GA).
