  The integration of renewable energy sources (RES) is rapidly increasing in
electric power systems (EPS). While the inclusion of intermittent RES coupled
with the wide-scale deployment of communication and sensing devices is
important towards a fully smart grid, it has also expanded the cyber-threat
landscape, effectively making power systems vulnerable to cyberattacks. This
paper proposes a cybersecurity assessment approach designed to assess the
cyberphysical security of EPS. The work takes into consideration the
intermittent generation of RES, vulnerabilities introduced by
microprocessor-based electronic information and operational technology (IT/OT)
devices, and contingency analysis results. The proposed approach utilizes deep
reinforcement learning (DRL) and an adapted Common Vulnerability Scoring System
(CVSS) score tailored to assess vulnerabilities in EPS in order to identify the
optimal attack transition policy based on N-2 contingency results, i.e., the
simultaneous failure of two system elements. The effectiveness of the work is
validated via numerical and real-time simulation experiments performed on
literature-based power grid test cases. The results demonstrate how the
proposed method based on deep Q-network (DQN) performs closely to a
graph-search approach in terms of the number of transitions needed to find the
optimal attack policy, without the need for full observation of the system. In
addition, the experiments present the method's scalability by showcasing the
number of transitions needed to find the optimal attack transition policy in a
large system such as the Polish 2383 bus test system. The results exhibit how
the proposed approach requires one order of magnitude fewer transitions when
compared to a random transition policy.
