  The long-tail distribution of real driving data poses challenges for training
and testing autonomous vehicles (AV), where rare yet crucial safety-critical
scenarios are infrequent. And virtual simulation offers a low-cost and
efficient solution. This paper proposes a near-miss focused training framework
for AV. Utilizing the driving scenario information provided by sensors in the
simulator, we design novel reward functions, which enable background vehicles
(BV) to generate near-miss scenarios and ensure gradients exist not only in
collision-free scenes but also in collision scenarios. And then leveraging the
Robust Adversarial Reinforcement Learning (RARL) framework for simultaneous
training of AV and BV to gradually enhance AV and BV capabilities, as well as
generating near-miss scenarios tailored to different levels of AV capabilities.
Results from three testing strategies indicate that the proposed method
generates scenarios closer to near-miss, thus enhancing the capabilities of
both AVs and BVs throughout training.
