  Emergency vehicle (EMV) service is a key function of cities and is
exceedingly challenging due to urban traffic congestion. The main reason behind
EMV service delay is the lack of communication and cooperation between vehicles
blocking EMVs. In this paper, we study the improvement of EMV service under V2X
connectivity. We consider the establishment of dynamic queue jump lanes (DQJLs)
based on real-time coordination of connected vehicles in the presence of
non-connected human-driven vehicles. We develop a novel Markov decision process
formulation for the DQJL coordination strategies, which explicitly accounts for
the uncertainty of drivers' yielding pattern to approaching EMVs. Based on
pairs of neural networks representing actors and critics for agent vehicles, we
develop a multi-agent actor-critic deep reinforcement learning algorithm that
handles a varying number of vehicles and a random proportion of connected
vehicles in the traffic. Approaching the optimal coordination strategies via
indirect and direct reinforcement learning, we present two schemata to address
multi-agent reinforcement learning on this connected vehicle application. Both
approaches are validated, on a micro-simulation testbed SUMO, to establish a
DQJL fast and safely. Validation results reveal that, with DQJL coordination
strategies, it saves up to 30% time for EMVs to pass a link-level intelligent
urban roadway than the baseline scenario.
