  Smart traffic control and management become an emerging application for Deep
Reinforcement Learning (DRL) to solve traffic congestion problems in urban
networks. Different traffic control and management policies can be tested on
the traffic simulation. Current DRL-based studies are mainly supported by the
microscopic simulation software (e.g., SUMO), while it is not suitable for
city-wide control due to the computational burden and gridlock effect. To the
best of our knowledge, there is a lack of studies on the large-scale traffic
simulator for DRL testbeds, which could further hinder the development of DRL.
In view of this, we propose a meso-macro traffic simulator for very large-scale
DRL scenarios. The proposed simulator integrates mesoscopic and macroscopic
traffic simulation models to improve efficiency and eliminate gridlocks. The
mesoscopic link model simulates flow dynamics on roads, and the macroscopic
Bathtub model depicts vehicle movement in regions. Moreover, both types of
models can be hybridized to accommodate various DRL tasks. This creates portals
for mixed transportation applications under different contexts. The result
shows that the developed simulator only takes 46 seconds to finish a 24-hour
simulation in a very large city with 2.2 million vehicles, which is much faster
than SUMO. Additionally, we develop a graphic interface for users to visualize
the simulation results in a web explorer. In the future, the developed
meso-macro traffic simulator could serve as a new environment for very
large-scale DRL problems.
