  Today, human operators primarily perform voltage control of the electric
transmission system. As the complexity of the grid increases, so does its
operation, suggesting additional automation could be beneficial. A subset of
machine learning known as deep reinforcement learning (DRL) has recently shown
promise in performing tasks typically performed by humans. This paper applies
DRL to the transmission voltage control problem, presents open-source DRL
environments for voltage control, proposes a novel modification to the "deep Q
network" (DQN) algorithm, and performs experiments at scale with systems up to
500 buses. The promise of applying DRL to voltage control is demonstrated,
though more research is needed to enable DRL-based techniques to consistently
outperform conventional methods.
