  Developing safe human-robot interaction systems is a necessary step towards
the widespread integration of autonomous agents in society. A key component of
such systems is the ability to reason about the many potential futures (e.g.
trajectories) of other agents in the scene. Towards this end, we present the
Trajectron, a graph-structured model that predicts many potential future
trajectories of multiple agents simultaneously in both highly dynamic and
multimodal scenarios (i.e. where the number of agents in the scene is
time-varying and there are many possible highly-distinct futures for each
agent). It combines tools from recurrent sequence modeling and variational deep
generative modeling to produce a distribution of future trajectories for each
agent in a scene. We demonstrate the performance of our model on several
datasets, obtaining state-of-the-art results on standard trajectory prediction
metrics as well as introducing a new metric for comparing models that output
distributions.
