  Reinforcement learning-based (RL-based) energy management strategy (EMS) is
considered a promising solution for the energy management of electric vehicles
with multiple power sources. It has been shown to outperform conventional
methods in energy management problems regarding energy-saving and real-time
performance. However, previous studies have not systematically examined the
essential elements of RL-based EMS. This paper presents an empirical analysis
of RL-based EMS in a Plug-in Hybrid Electric Vehicle (PHEV) and Fuel Cell
Electric Vehicle (FCEV). The empirical analysis is developed in four aspects:
algorithm, perception and decision granularity, hyperparameters, and reward
function. The results show that the Off-policy algorithm effectively develops a
more fuel-efficient solution within the complete driving cycle compared with
other algorithms. Improving the perception and decision granularity does not
produce a more desirable energy-saving solution but better balances battery
power and fuel consumption. The equivalent energy optimization objective based
on the instantaneous state of charge (SOC) variation is parameter sensitive and
can help RL-EMSs to achieve more efficient energy-cost strategies.
