  Multi-agent control is a central theme in the Cyber-Physical Systems (CPS).
However, current control methods either receive non-Markovian states due to
insufficient sensing and decentralized design, or suffer from poor convergence.
This paper presents the Delayed Propagation Transformer (DePT), a new
transformer-based model that specializes in the global modeling of CPS while
taking into account the immutable constraints from the physical world. DePT
induces a cone-shaped spatial-temporal attention prior, which injects the
information propagation and aggregation principles and enables a global view.
With physical constraint inductive bias baked into its design, our DePT is
ready to plug and play for a broad class of multi-agent systems. The
experimental results on one of the most challenging CPS -- network-scale
traffic signal control system in the open world -- show that our model
outperformed the state-of-the-art expert methods on synthetic and real-world
datasets. Our codes are released at: https://github.com/VITA-Group/DePT.
