  This paper considers the sequential design of remedial control actions in
response to system anomalies for the ultimate objective of preventing
blackouts. A physics-guided reinforcement learning (RL) framework is designed
to identify effective sequences of real-time remedial look-ahead decisions
accounting for the long-term impact on the system's stability. The paper
considers a space of control actions that involve both discrete-valued
transmission line-switching decisions (line reconnections and removals) and
continuous-valued generator adjustments. To identify an effective blackout
mitigation policy, a physics-guided approach is designed that uses power-flow
sensitivity factors associated with the power transmission network to guide the
RL exploration during agent training. Comprehensive empirical evaluations using
the open-source Grid2Op platform demonstrate the notable advantages of
incorporating physical signals into RL decisions, establishing the gains of the
proposed physics-guided approach compared to its black box counterparts. One
important observation is that strategically~\emph{removing} transmission lines,
in conjunction with multiple real-time generator adjustments, often renders
effective long-term decisions that are likely to prevent or delay blackouts.
