  Trajectory prediction is essential for autonomous vehicles (AVs) to plan
correct and safe driving behaviors. While many prior works aim to achieve
higher prediction accuracy, few study the adversarial robustness of their
methods. To bridge this gap, we propose to study the adversarial robustness of
data-driven trajectory prediction systems. We devise an optimization-based
adversarial attack framework that leverages a carefully-designed differentiable
dynamic model to generate realistic adversarial trajectories. Empirically, we
benchmark the adversarial robustness of state-of-the-art prediction models and
show that our attack increases the prediction error for both general metrics
and planning-aware metrics by more than 50% and 37%. We also show that our
attack can lead an AV to drive off road or collide into other vehicles in
simulation. Finally, we demonstrate how to mitigate the adversarial attacks
using an adversarial training scheme.
