  Autonomous vehicles ought to predict the surrounding agents' trajectories to
allow safe maneuvers in uncertain and complex traffic situations. As companies
increasingly apply trajectory prediction in the real world, security becomes a
relevant concern. In this paper, we focus on backdoors - a security threat
acknowledged in other fields but so far overlooked for trajectory prediction.
To this end, we describe and investigate four triggers that could affect
trajectory prediction. We then show that these triggers (for example, a braking
vehicle), when correlated with a desired output (for example, a curve) during
training, cause the desired output of a state-of-the-art trajectory prediction
model. In other words, the model has good benign performance but is vulnerable
to backdoors. This is the case even if the trigger maneuver is performed by a
non-casual agent behind the target vehicle. As a side-effect, our analysis
reveals interesting limitations within trajectory prediction models. Finally,
we evaluate a range of defenses against backdoors. While some, like simple
offroad checks, do not enable detection for all triggers, clustering is a
promising candidate to support manual inspection to find backdoors.
