  The increasing integration of electric vehicles (EVs) into the grid can pose
a significant risk to the distribution system operation in the absence of
coordination. In response to the need for effective coordination of EVs within
the distribution network, this paper presents a safety-aware reinforcement
learning (RL) algorithm designed to manage EV charging stations while ensuring
the satisfaction of system constraints. Unlike existing methods, our proposed
algorithm does not rely on explicit penalties for constraint violations,
eliminating the need for penalty coefficient tuning. Furthermore, managing EV
charging stations is further complicated by multiple uncertainties, notably the
variability in solar energy generation and energy prices. To address this
challenge, we develop an off-policy RL algorithm to efficiently utilize data to
learn patterns in such uncertain environments. Our algorithm also incorporates
a maximum entropy framework to enhance the RL algorithm's exploratory process,
preventing convergence to local optimal solutions. Simulation results
demonstrate that our algorithm outperforms traditional RL algorithms in
managing EV charging in the distribution network.
