  We took part in the city brain challenge competition and achieved the 8th
place. In this competition, the players are provided with a real-world
city-scale road network and its traffic demand derived from real traffic data.
The players are asked to coordinate the traffic signals with a self-designed
agent to maximize the number of vehicles served while maintaining an acceptable
delay. In this abstract paper, we present an overall analysis and our detailed
solution to this competition. Our approach is mainly based on the adaptation of
the deep Q-network (DQN) for real-time traffic signal control. From our
perspective, the major challenge of this competition is how to extend the
classical DQN framework to traffic signals control in real-world complex road
network and traffic flow situation. After trying and implementing several
classical reward functions, we finally chose to apply our newly-designed reward
in our agent. By applying our newly-proposed reward function and carefully
tuning the control scheme, an agent based on a single DQN model can rank among
the top 15 teams. We hope this paper could serve, to some extent, as a baseline
solution to traffic signal control of real-world road network and inspire
further attempts and researches.
