  With the increase of connectivity in power grid, a cascading failure may be
triggered by the failure of a transmission line, which can lead to substantial
economic losses and serious negative social impacts. Therefore, it is very
important to identify the critical lines under various types of attacks that
may initiate a cascading failure and deploy defense resources to protect them.
Since coordinated multistage line attacks can lead to larger negative impacts
compared with a single-stage attack or a multistage attack without
coordination, this paper intends to identify the critical lines under
coordinated multistage attacks that may initiate a cascading failure and deploy
limited defense resources optimally. To this end, we first formulate a total
generation loss maximization problem with the consideration of multiple
attackers and multiple stages. Due to the large size of solution space, it is
very challenging to solve the formulated problem. To overcome the challenge, we
reformulate the problem as a Markov game and design its components, e.g.,
state, action, and reward. Next, we propose a scalable algorithm to solve the
Markov game based on multi-agent deep reinforcement learning and prioritized
experience replay, which can determine the optimal attacking line sequences.
Then, we design a defense strategy to decide the optimal defense line set.
Extensive simulation results show the effectiveness of the proposed algorithm
and the designed defense strategy.
