  The recent advancements in wireless technology enable connected autonomous
vehicles (CAVs) to gather information about their environment by
vehicle-to-vehicle (V2V) communication. In this work, we design an
information-sharing-based multi-agent reinforcement learning (MARL) framework
for CAVs, to take advantage of the extra information when making decisions to
improve traffic efficiency and safety. The safe actor-critic algorithm we
propose has two new techniques: the truncated Q-function and safe action
mapping. The truncated Q-function utilizes the shared information from
neighboring CAVs such that the joint state and action spaces of the Q-function
do not grow in our algorithm for a large-scale CAV system. We prove the bound
of the approximation error between the truncated-Q and global Q-functions. The
safe action mapping provides a provable safety guarantee for both the training
and execution based on control barrier functions. Using the CARLA simulator for
experiments, we show that our approach can improve the CAV system's efficiency
in terms of average velocity and comfort under different CAV ratios and
different traffic densities. We also show that our approach avoids the
execution of unsafe actions and always maintains a safe distance from other
vehicles. We construct an obstacle-at-corner scenario to show that the shared
vision can help CAVs to observe obstacles earlier and take action to avoid
traffic jams.
