  In this paper, we propose THOMAS, a joint multi-agent trajectory prediction
framework allowing for an efficient and consistent prediction of multi-agent
multi-modal trajectories. We present a unified model architecture for
simultaneous agent future heatmap estimation, in which we leverage hierarchical
and sparse image generation for fast and memory-efficient inference. We propose
a learnable trajectory recombination model that takes as input a set of
predicted trajectories for each agent and outputs its consistent reordered
recombination. This recombination module is able to realign the initially
independent modalities so that they do no collide and are coherent with each
other. We report our results on the Interaction multi-agent prediction
challenge and rank $1^{st}$ on the online test leaderboard.
