  Urban Traffic Control (UTC) plays an essential role in Intelligent
Transportation System (ITS) but remains difficult. Since model-based UTC
methods may not accurately describe the complex nature of traffic dynamics in
all situations, model-free data-driven UTC methods, especially reinforcement
learning (RL) based UTC methods, received increasing interests in the last
decade. However, existing DL approaches did not propose an efficient algorithm
to solve the complicated multiple intersections control problems whose
state-action spaces are vast. To solve this problem, we propose a Deep
Reinforcement Learning (DRL) algorithm that combines several tricks to master
an appropriate control strategy within an acceptable time. This new algorithm
relaxes the fixed traffic demand pattern assumption and reduces human invention
in parameter tuning. Simulation experiments have shown that our method
outperforms traditional rule-based approaches and has the potential to handle
more complex traffic problems in the real world.
