  This paper considers optimal traffic signal control in smart cities, which
has been taken as a complex networked system control problem. Given the
interacting dynamics among traffic lights and road networks, attaining
controller adaptivity and scalability stands out as a primary challenge.
Capturing the spatial-temporal correlation among traffic lights under the
framework of Multi-Agent Reinforcement Learning (MARL) is a promising solution.
Nevertheless, existing MARL algorithms ignore effective information aggregation
which is fundamental for improving the learning capacity of decentralized
agents. In this paper, we design a new decentralized control architecture with
improved environmental observability to capture the spatial-temporal
correlation. Specifically, we first develop a topology-aware information
aggregation strategy to extract correlation-related information from
unstructured data gathered in the road network. Particularly, we transfer the
road network topology into a graph shift operator by forming a diffusion
process on the topology, which subsequently facilitates the construction of
graph signals. A diffusion convolution module is developed, forming a new MARL
algorithm, which endows agents with the capabilities of graph learning.
Extensive experiments based on both synthetic and real-world datasets verify
that our proposal outperforms existing decentralized algorithms.
