  Since hydrogen has many advantages (e.g., free pollution, extensive sources,
convenient storage and transportation), hydrogen-based multi-energy systems
(HMESs) have received wide attention. However, existing works on the optimal
operation of HMESs neglect building thermal dynamics, which means that the
flexibility of building thermal loads can not be utilized for reducing system
operation cost. In this paper, we investigate an optimal operation problem of
an HMES with the consideration of building thermal dynamics. Specifically, we
first formulate an expected operational cost minimization problem related to an
HMES. Due to the existence of uncertain parameters, inexplicit building thermal
dynamics models, temporally coupled operational constraints related to three
kinds of energy storage systems and indoor temperatures, as well as the
coupling between electric energy subsystems and thermal energy subsystems, it
is challenging to solve the formulated problem. To overcome the challenge, we
reformulate the problem as a Markov game and propose an energy management
algorithm to solve it based on multi-agent discrete actor-critic with rules
(MADACR). Note that the proposed algorithm does not require any prior knowledge
of uncertain parameters, parameter prediction, and explicit building thermal
dynamics model. Simulation results based on real-world traces show the
effectiveness of the proposed algorithm.
