  Energy management systems (EMS) are becoming increasingly important in order
to utilize the continuously growing curtailed renewable energy. Promising
energy storage systems (ESS), such as batteries and green hydrogen should be
employed to maximize the efficiency of energy stakeholders. However, optimal
decision-making, i.e., planning the leveraging between different strategies, is
confronted with the complexity and uncertainties of large-scale problems. Here,
we propose a sophisticated deep reinforcement learning (DRL) methodology with a
policy-based algorithm to realize the real-time optimal ESS planning under the
curtailed renewable energy uncertainty. A quantitative performance comparison
proved that the DRL agent outperforms the scenario-based stochastic
optimization (SO) algorithm, even with a wide action and observation space.
Owing to the uncertainty rejection capability of the DRL, we could confirm a
robust performance, under a large uncertainty of the curtailed renewable
energy, with a maximizing net profit and stable system. Action-mapping was
performed for visually assessing the action taken by the DRL agent according to
the state. The corresponding results confirmed that the DRL agent learns the
way like what a human expert would do, suggesting reliable application of the
proposed methodology.
