  The intelligent control of the traffic signal is critical to the optimization
of transportation systems. To achieve global optimal traffic efficiency in
large-scale road networks, recent works have focused on coordination among
intersections, which have shown promising results. However, existing studies
paid more attention to observations sharing among intersections (both explicit
and implicit) and did not care about the consequences after decisions. In this
paper, we design a multiagent coordination framework based on Deep
Reinforcement Learning methods for traffic signal control, defined as
{\gamma}-Reward that includes both original {\gamma}-Reward and
{\gamma}-Attention-Reward. Specifically, we propose the Spatial Differentiation
method for coordination which uses the temporal-spatial information in the
replay buffer to amend the reward of each action. A concise theoretical
analysis that proves the proposed model can converge to Nash equilibrium is
given. By extending the idea of Markov Chain to the dimension of space-time,
this truly decentralized coordination mechanism replaces the graph attention
method and realizes the decoupling of the road network, which is more scalable
and more in line with practice. The simulation results show that the proposed
model remains a state-of-the-art performance even not use a centralized
setting. Code is available in https://github.com/Skylark0924/Gamma Reward.
