  We introduce a Deep Stochastic IOC RNN Encoderdecoder framework, DESIRE, for
the task of future predictions of multiple interacting agents in dynamic
scenes. DESIRE effectively predicts future locations of objects in multiple
scenes by 1) accounting for the multi-modal nature of the future prediction
(i.e., given the same context, future may vary), 2) foreseeing the potential
future outcomes and make a strategic prediction based on that, and 3) reasoning
not only from the past motion history, but also from the scene context as well
as the interactions among the agents. DESIRE achieves these in a single
end-to-end trainable neural network model, while being computationally
efficient. The model first obtains a diverse set of hypothetical future
prediction samples employing a conditional variational autoencoder, which are
ranked and refined by the following RNN scoring-regression module. Samples are
scored by accounting for accumulated future rewards, which enables better
long-term strategic decisions similar to IOC frameworks. An RNN scene context
fusion module jointly captures past motion histories, the semantic scene
context and interactions among multiple agents. A feedback mechanism iterates
over the ranking and refinement to further boost the prediction accuracy. We
evaluate our model on two publicly available datasets: KITTI and Stanford Drone
Dataset. Our experiments show that the proposed model significantly improves
the prediction accuracy compared to other baseline methods.
