  With the advances in the Internet of Things technology, electric vehicles
(EVs) have become easier to schedule in daily life, which is reshaping the
electric load curve. It is important to design efficient charging algorithms to
mitigate the negative impact of EV charging on the power grid. This paper
investigates an EV charging scheduling problem to reduce the charging cost
while shaving the peak charging load, under unknown future information about
EVs, such as arrival time, departure time, and charging demand. First, we
formulate an EV charging problem to minimize the electricity bill of the EV
fleet and study the EV charging problem in an online setting without knowing
future information. We develop an actor-critic learning-based smart charging
algorithm (SCA) to schedule the EV charging against the uncertainties in EV
charging behaviors. The SCA learns an optimal EV charging strategy with
continuous charging actions instead of discrete approximation of charging. We
further develop a more computationally efficient customized actor-critic
learning charging algorithm (CALC) by reducing the state dimension and thus
improving the computational efficiency. Finally, simulation results show that
our proposed SCA can reduce EVs' expected cost by 24.03%, 21.49%, 13.80%,
compared with the Eagerly Charging Algorithm, Online Charging Algorithm,
RL-based Adaptive Energy Management Algorithm, respectively. CALC is more
computationally efficient, and its performance is close to that of SCA with
only a gap of 5.56% in the cost.
