  Future prediction is a fundamental principle of intelligence that helps plan
actions and avoid possible dangers. As the future is uncertain to a large
extent, modeling the uncertainty and multimodality of the future states is of
great relevance. Existing approaches are rather limited in this regard and
mostly yield a single hypothesis of the future or, at the best, strongly
constrained mixture components that suffer from instabilities in training and
mode collapse. In this work, we present an approach that involves the
prediction of several samples of the future with a winner-takes-all loss and
iterative grouping of samples to multiple modes. Moreover, we discuss how to
evaluate predicted multimodal distributions, including the common real
scenario, where only a single sample from the ground-truth distribution is
available for evaluation. We show on synthetic and real data that the proposed
approach triggers good estimates of multimodal distributions and avoids mode
collapse. Source code is available at
$\href{https://github.com/lmb-freiburg/Multimodal-Future-Prediction}{\text{this
https URL.}}$
