  This article proposes a novel approach to traffic signal control that
combines phase re-service with reinforcement learning (RL). The RL agent
directly determines the duration of the next phase in a pre-defined sequence.
Before the RL agent's decision is executed, we use the shock wave theory to
estimate queue expansion at the designated movement allowed for re-service and
decide if phase re-service is necessary. If necessary, a temporary phase
re-service is inserted before the next regular phase. We formulate the RL
problem as a semi-Markov decision process (SMDP) and solve it with proximal
policy optimization (PPO). We conducted a series of experiments that showed
significant improvements thanks to the introduction of phase re-service.
Vehicle delays are reduced by up to 29.95% of the average and up to 59.21% of
the standard deviation. The number of stops is reduced by 26.05% on average
with 45.77% less standard deviation.
