  With the recent advances in mobile energy storage technologies, electric
vehicles (EVs) have become a crucial part of smart grids. When EVs participate
in the demand response program, the charging cost can be significantly reduced
by taking full advantage of the real-time pricing signals. However, many
stochastic factors exist in the dynamic environment, bringing significant
challenges to design an optimal charging/discharging control strategy. This
paper develops an optimal EV charging/discharging control strategy for
different EV users under dynamic environments to maximize EV users' benefits.
We first formulate this problem as a Markov decision process (MDP). Then we
consider EV users with different behaviors as agents in different environments.
Furthermore, a horizontal federated reinforcement learning (HFRL)-based method
is proposed to fit various users' behaviors and dynamic environments. This
approach can learn an optimal charging/discharging control strategy without
sharing users' profiles. Simulation results illustrate that the proposed
real-time EV charging/discharging control strategy can perform well among
various stochastic factors.
