  This paper proposes a reinforcement learning approach for traffic control
with the adaptive horizon. To build the controller for the traffic network, a
Q-learning-based strategy that controls the green light passing time at the
network intersections is applied. The controller includes two components: the
regular Q-learning controller that controls the traffic light signal, and the
adaptive controller that continuously optimizes the action space for the
Q-learning algorithm in order to improve the efficiency of the Q-learning
algorithm. The regular Q-learning controller uses the control cost function as
a reward function to determine the action to choose. The adaptive controller
examines the control cost and updates the action space of the controller by
determining the subset of actions that are most likely to obtain optimal
results and shrinking the action space to that subset. Uncertainties in traffic
influx and turning rate are introduced to test the robustness of the controller
under a stochastic environment. Compared with those with model predictive
control (MPC), the results show that the proposed Q-learning-based controller
outperforms the MPC method by reaching a stable solution in a shorter period
and achieves lower control costs. The proposed Q-learning-based controller is
also robust under 30% traffic demand uncertainty and 15% turning rate
uncertainty.
