  Generating safety-critical scenarios is essential for testing and verifying
the safety of autonomous vehicles. Traditional optimization techniques suffer
from the curse of dimensionality and limit the search space to fixed parameter
spaces. To address these challenges, we propose a deep reinforcement learning
approach that generates scenarios by sequential editing, such as adding new
agents or modifying the trajectories of the existing agents. Our framework
employs a reward function consisting of both risk and plausibility objectives.
The plausibility objective leverages generative models, such as a variational
autoencoder, to learn the likelihood of the generated parameters from the
training datasets; It penalizes the generation of unlikely scenarios. Our
approach overcomes the dimensionality challenge and explores a wide range of
safety-critical scenarios. Our evaluation demonstrates that the proposed method
generates safety-critical scenarios of higher quality compared with previous
approaches.
