  Acquiring driving policies that can transfer to unseen environments is
challenging when driving in dense traffic flows. The design of traffic flow is
essential and previous studies are unable to balance interaction and
safety-criticism. To tackle this problem, we propose a socially adversarial
traffic flow. We propose a Contextual Partially-Observable Stochastic Game to
model traffic flow and assign Social Value Orientation (SVO) as context. We
then adopt a two-stage framework. In Stage 1, each agent in our socially-aware
traffic flow is driven by a hierarchical policy where upper-level policy
communicates genuine SVOs of all agents, which the lower-level policy takes as
input. In Stage 2, each agent in the socially adversarial traffic flow is
driven by the hierarchical policy where upper-level communicates mistaken SVOs,
taken by the lower-level policy trained in Stage 1. Driving policy is
adversarially trained through a zero-sum game formulation with upper-level
policies, resulting in a policy with enhanced zero-shot transfer capability to
unseen traffic flows. Comprehensive experiments on cross-validation verify the
superior zero-shot transfer performance of our method.
