  Communication technologies enable coordination among connected and autonomous
vehicles (CAVs). However, it remains unclear how to utilize shared information
to improve the safety and efficiency of the CAV system in dynamic and
complicated driving scenarios. In this work, we propose a framework of
constrained multi-agent reinforcement learning (MARL) with a parallel Safety
Shield for CAVs in challenging driving scenarios that includes unconnected
hazard vehicles. The coordination mechanisms of the proposed MARL include
information sharing and cooperative policy learning, with Graph Convolutional
Network (GCN)-Transformer as a spatial-temporal encoder that enhances the
agent's environment awareness. The Safety Shield module with Control Barrier
Functions (CBF)-based safety checking protects the agents from taking unsafe
actions. We design a constrained multi-agent advantage actor-critic (CMAA2C)
algorithm to train safe and cooperative policies for CAVs. With the experiment
deployed in the CARLA simulator, we verify the performance of the safety
checking, spatial-temporal encoder, and coordination mechanisms designed in our
method by comparative experiments in several challenging scenarios with
unconnected hazard vehicles. Results show that our proposed methodology
significantly increases system safety and efficiency in challenging scenarios.
