  Proper functioning of connected and automated vehicles (CAVs) is crucial for
the safety and efficiency of future intelligent transport systems. Meanwhile,
transitioning to fully autonomous driving requires a long period of mixed
autonomy traffic, including both CAVs and human-driven vehicles. Thus,
collaboration decision-making for CAVs is essential to generate appropriate
driving behaviors to enhance the safety and efficiency of mixed autonomy
traffic. In recent years, deep reinforcement learning (DRL) has been widely
used in solving decision-making problems. However, the existing DRL-based
methods have been mainly focused on solving the decision-making of a single
CAV. Using the existing DRL-based methods in mixed autonomy traffic cannot
accurately represent the mutual effects of vehicles and model dynamic traffic
environments. To address these shortcomings, this article proposes a graph
reinforcement learning (GRL) approach for multi-agent decision-making of CAVs
in mixed autonomy traffic. First, a generic and modular GRL framework is
designed. Then, a systematic review of DRL and GRL methods is presented,
focusing on the problems addressed in recent research. Moreover, a comparative
study on different GRL methods is further proposed based on the designed
framework to verify the effectiveness of GRL methods. Results show that the GRL
methods can well optimize the performance of multi-agent decision-making for
CAVs in mixed autonomy traffic compared to the DRL methods. Finally, challenges
and future research directions are summarized. This study can provide a
valuable research reference for solving the multi-agent decision-making
problems of CAVs in mixed autonomy traffic and can promote the implementation
of GRL-based methods into intelligent transportation systems. The source code
of our work can be found at https://github.com/Jacklinkk/Graph_CAVs.
