  The emergence of reinforcement learning (RL) methods in traffic signal
control tasks has achieved better performance than conventional rule-based
approaches. Most RL approaches require the observation of the environment for
the agent to decide which action is optimal for a long-term reward. However, in
real-world urban scenarios, missing observation of traffic states may
frequently occur due to the lack of sensors, which makes existing RL methods
inapplicable on road networks with missing observation. In this work, we aim to
control the traffic signals in a real-world setting, where some of the
intersections in the road network are not installed with sensors and thus with
no direct observations around them. To the best of our knowledge, we are the
first to use RL methods to tackle the traffic signal control problem in this
real-world setting. Specifically, we propose two solutions: the first one
imputes the traffic states to enable adaptive control, and the second one
imputes both states and rewards to enable adaptive control and the training of
RL agents. Through extensive experiments on both synthetic and real-world road
network traffic, we reveal that our method outperforms conventional approaches
and performs consistently with different missing rates. We also provide further
investigations on how missing data influences the performance of our model.
