  The widespread adoption of electric vehicles (EVs) poses several challenges
to power distribution networks and smart grid infrastructure due to the
possibility of significantly increasing electricity demands, especially during
peak hours. Furthermore, when EVs participate in demand-side management
programs, charging expenses can be reduced by using optimal charging control
policies that fully utilize real-time pricing schemes. However, devising
optimal charging methods and control strategies for EVs is challenging due to
various stochastic and uncertain environmental factors. Currently, most EV
charging controllers operate based on a centralized model. In this paper, we
introduce a novel approach for distributed and cooperative charging strategy
using a Multi-Agent Reinforcement Learning (MARL) framework. Our method is
built upon the Deep Deterministic Policy Gradient (DDPG) algorithm for a group
of EVs in a residential community, where all EVs are connected to a shared
transformer. This method, referred to as CTDE-DDPG, adopts a Centralized
Training Decentralized Execution (CTDE) approach to establish cooperation
between agents during the training phase, while ensuring a distributed and
privacy-preserving operation during execution. We theoretically examine the
performance of centralized and decentralized critics for the DDPG-based MARL
implementation and demonstrate their trade-offs. Furthermore, we numerically
explore the efficiency, scalability, and performance of centralized and
decentralized critics. Our theoretical and numerical results indicate that,
despite higher policy gradient variances and training complexity, the CTDE-DDPG
framework significantly improves charging efficiency by reducing total
variation by approximately %36 and charging cost by around %9.1 on average...
