  In this article, we demonstrate a zero-shot transfer of an autonomous driving
policy from simulation to University of Delaware's scaled smart city with
adversarial multi-agent reinforcement learning, in which an adversary attempts
to decrease the net reward by perturbing both the inputs and outputs of the
autonomous vehicles during training. We train the autonomous vehicles to
coordinate with each other while crossing a roundabout in the presence of an
adversary in simulation. The adversarial policy successfully reproduces the
simulated behavior and incidentally outperforms, in terms of travel time, both
a human-driving baseline and adversary-free trained policies. Finally, we
demonstrate that the addition of adversarial training considerably improves the
performance \eat{stability and robustness} of the policies after transfer to
the real world compared to Gaussian noise injection.
