  In recent years, there has been increasing amount of interest around meta
reinforcement learning methods for traffic signal control, which have achieved
better performance compared with traditional control methods. However, previous
methods lack robustness in adaptation and stability in training process in
complex situations, which largely limits its application in real-world traffic
signal control. In this paper, we propose a novel value-based Bayesian
meta-reinforcement learning framework BM-DQN to robustly speed up the learning
process in new scenarios by utilizing well-trained prior knowledge learned from
existing scenarios. This framework is based on our proposed fast-adaptation
variation to Gradient-EM Bayesian Meta-learning and the fast-update advantage
of DQN, which allows for fast adaptation to new scenarios with continual
learning ability and robustness to uncertainty. The experiments on restricted
2D navigation and traffic signal control show that our proposed framework
adapts more quickly and robustly in new scenarios than previous methods, and
specifically, much better continual learning ability in heterogeneous
scenarios.
