  Owing to large industrial energy consumption, industrial production has
brought a huge burden to the grid in terms of renewable energy access and power
supply. Due to the coupling of multiple energy sources and the uncertainty of
renewable energy and demand, centralized methods require large calculation and
coordination overhead. Thus, this paper proposes a multi-energy management
framework achieved by decentralized execution and centralized training for an
industrial park. The energy management problem is formulated as a
partially-observable Markov decision process, which is intractable by dynamic
programming due to the lack of the prior knowledge of the underlying stochastic
process. The objective is to minimize long-term energy costs while ensuring the
demand of users. To solve this issue and improve the calculation speed, a novel
multi-agent deep reinforcement learning algorithm is proposed, which contains
the following key points: counterfactual baseline for facilitating contributing
agents to learn better policies, soft actor-critic for improving robustness and
exploring optimal solutions. A novel reward is designed by Lagrange multiplier
method to ensure the capacity constraints of energy storage. In addition,
considering that the increase in the number of agents leads to performance
degradation due to large observation spaces, an attention mechanism is
introduced to enhance the stability of policy and enable agents to focus on
important energy-related information, which improves the exploration efficiency
of soft actor-critic. Numerical results based on actual data verify the
performance of the proposed algorithm with high scalability, indicating that
the industrial park can minimize energy costs under different demands.
