  In replacing fossil fuels with renewable energy resources for carbon
neutrality, the unbalanced resource production of intermittent wind and
photovoltaic (PV) power is a critical issue for peer-to-peer (P2P) power
trading. To address this issue, a reinforcement learning (RL) technique is
introduced in this paper. For RL, a graph convolutional network (GCN) and a
bi-directional long short-term memory (Bi-LSTM) network are jointly applied to
P2P power trading between nanogrid clusters, based on cooperative game theory.
The flexible and reliable DC nanogrid is suitable for integrating renewable
energy for a distribution system. Each local nanogrid cluster takes the
position of prosumer, focusing on power production and consumption
simultaneously. For the power management of nanogrid cluster, multi-objective
optimization is applied to each local nanogrid cluster with the Internet of
Things (IoT) technology. Charging/discharging of an electric vehicle (EV) is
executed considering the intermittent characteristics of wind and PV power
production. RL algorithms, such as GCN- convolutional neural network (CNN)
layers for deep Q-learning network (DQN), GCN-LSTM layers for deep recurrent
Q-learning network (DRQN), GCN-Bi-LSTM layers for DRQN, and GCN-Bi-LSTM layers
for proximal policy optimization (PPO), are used for simulations. Consequently,
the cooperative P2P power trading system maximizes the profit by considering
the time of use (ToU) tariff-based electricity cost and the system marginal
price (SMP), and minimizes the amount of grid power consumption. Power
management of nanogrid clusters with P2P power trading is simulated on a
distribution test feeder in real time, and the proposed GCN-Bi-LSTM-PPO
technique achieving the lowest electricity cost among the RL algorithms used
for comparison reduces the electricity cost by 36.7%, averaging over nanogrid
clusters.
