  Naturalistic driving trajectories are crucial for the performance of
autonomous driving algorithms. However, most of the data is collected in safe
scenarios leading to the duplication of trajectories which are easy to be
handled by currently developed algorithms. When considering safety, testing
algorithms in near-miss scenarios that rarely show up in off-the-shelf datasets
is a vital part of the evaluation. As a remedy, we propose a near-miss data
synthesizing framework based on Variational Bayesian methods and term it as
Conditional Multiple Trajectory Synthesizer (CMTS). We leverage a generative
model conditioned on road maps to bridge safe and collision driving data by
representing their distribution in the latent space. By sampling from the
near-miss distribution, we can synthesize safety-critical data crucial for
understanding traffic scenarios but not shown in neither the original dataset
nor the collision dataset. Our experimental results demonstrate that the
augmented dataset covers more kinds of driving scenarios, especially the
near-miss ones, which help improve the trajectory prediction accuracy and the
capability of dealing with risky driving scenarios.
