  Reinforcement learning (RL) techniques for traffic signal control (TSC) have
gained increasing popularity in recent years. However, most existing RL-based
TSC methods tend to focus primarily on the RL model structure while neglecting
the significance of proper traffic state representation. Furthermore, some
RL-based methods heavily rely on expert-designed traffic signal phase
competition. In this paper, we present a novel approach to TSC that utilizes
queue length as an efficient state representation. We propose two new methods:
(1) Max Queue-Length (M-QL), an optimization-based traditional method designed
based on the property of queue length; and (2) AttentionLight, an RL model that
employs the self-attention mechanism to capture the signal phase correlation
without requiring human knowledge of phase relationships. Comprehensive
experiments on multiple real-world datasets demonstrate the effectiveness of
our approach: (1) the M-QL method outperforms the latest RL-based methods; (2)
AttentionLight achieves a new state-of-the-art performance; and (3) our results
highlight the significance of proper state representation, which is as crucial
as neural network design in TSC methods. Our findings have important
implications for advancing the development of more effective and efficient TSC
methods. Our code is released on Github (https://github.
com/LiangZhang1996/AttentionLight).
