  The effectiveness of traffic light control has been significantly improved by
current reinforcement learning-based approaches via better cooperation among
multiple traffic lights. However, a persisting issue remains: how to obtain a
multi-agent traffic signal control algorithm with remarkable transferability
across diverse cities? In this paper, we propose a Transformer on Transformer
(TonT) model for cross-city meta multi-agent traffic signal control, named as
X-Light: We input the full Markov Decision Process trajectories, and the Lower
Transformer aggregates the states, actions, rewards among the target
intersection and its neighbors within a city, and the Upper Transformer learns
the general decision trajectories across different cities. This dual-level
approach bolsters the model's robust generalization and transferability.
Notably, when directly transferring to unseen scenarios, ours surpasses all
baseline methods with +7.91% on average, and even +16.3% in some cases,
yielding the best results.
