  Effective traffic control methods have great potential in alleviating network
congestion. Existing literature generally focuses on a single control approach,
while few studies have explored the effectiveness of integrated and coordinated
control approaches. This study considers two representative control approaches:
ramp metering for freeways and perimeter control for homogeneous urban roads,
and we aim to develop a deep reinforcement learning (DRL)-based coordinated
control framework for large-scale networks. The main challenges are 1) there is
a lack of efficient dynamic models for both freeways and urban roads; 2) the
standard DRL method becomes ineffective due to the complex and non-stationary
network dynamics. In view of this, we propose a novel meso-macro dynamic
network model and first time develop a demonstration-guided DRL method to
achieve large-scale coordinated ramp metering and perimeter control. The
dynamic network model hybridizes the link and generalized bathtub models to
depict the traffic dynamics of freeways and urban roads, respectively. For the
DRL method, we incorporate demonstration to guide the DRL method for better
convergence by introducing the concept of "teacher" and "student" models. The
teacher models are traditional controllers (e.g., ALINEA, Gating), which
provide control demonstrations. The student models are DRL methods, which learn
from the teacher and aim to surpass the teacher's performance. To validate the
proposed framework, we conduct two case studies in a small-scale network and a
real-world large-scale traffic network in Hong Kong. The research outcome
reveals the great potential of combining traditional controllers with DRL for
coordinated control in large-scale networks.
