  Energy management strategy (EMS) is a key technology for plug-in hybrid
electric vehicles (PHEVs). The energy management of certain series-parallel
PHEVs involves the control of continuous variables, such as engine torque, and
discrete variables, such as clutch engagement/disengagement. We establish a
control-oriented model for a series-parallel plug-in hybrid system with clutch
engagement control from the perspective of mixed-integer programming.
Subsequently, we design an EMS based on continuous-discrete reinforcement
learning (CDRL), which enables simultaneous output of continuous and discrete
variables. During training, we introduce state-of-charge (SOC) randomization to
ensure that the hybrid system exhibits optimal energy-saving performance in
both high and low SOC. Finally, the effectiveness of the proposed CDRL strategy
is verified by comparing EMS based on charge-depleting charge-sustaining
(CD-CS) with rule-based clutch engagement control, and Dynamic Programming
(DP). The simulation results show that, under a high SOC, the CDRL strategy
proposed in this paper can improve energy efficiency by 8.3% compared to CD-CS,
and the energy consumption is just 6.6% higher than the global optimum based on
DP, while under a low SOC, the numbers are 4.1% and 3.9%, respectively.
