  With the adoption of autonomous vehicles on our roads, we will witness a
mixed-autonomy environment where autonomous and human-driven vehicles must
learn to co-exist by sharing the same road infrastructure. To attain
socially-desirable behaviors, autonomous vehicles must be instructed to
consider the utility of other vehicles around them in their decision-making
process. Particularly, we study the maneuver planning problem for autonomous
vehicles and investigate how a decentralized reward structure can induce
altruism in their behavior and incentivize them to account for the interest of
other autonomous and human-driven vehicles. This is a challenging problem due
to the ambiguity of a human driver's willingness to cooperate with an
autonomous vehicle. Thus, in contrast with the existing works which rely on
behavior models of human drivers, we take an end-to-end approach and let the
autonomous agents to implicitly learn the decision-making process of human
drivers only from experience. We introduce a multi-agent variant of the
synchronous Advantage Actor-Critic (A2C) algorithm and train agents that
coordinate with each other and can affect the behavior of human drivers to
improve traffic flow and safety.
