  Computer vision systems that are deployed in safety-critical applications
need to quantify their output uncertainty. We study regression from images to
parameter values and here it is common to detect uncertainty by predicting
probability distributions. In this context, we investigate the
regression-by-classification paradigm which can represent multimodal
distributions, without a prior assumption on the number of modes. Through
experiments on a specifically designed synthetic dataset, we demonstrate that
traditional loss functions lead to poor probability distribution estimates and
severe overconfidence, in the absence of full ground truth distributions. In
order to alleviate these issues, we propose hinge-Wasserstein -- a simple
improvement of the Wasserstein loss that reduces the penalty for weak secondary
modes during training. This enables prediction of complex distributions with
multiple modes, and allows training on datasets where full ground truth
distributions are not available. In extensive experiments, we show that the
proposed loss leads to substantially better uncertainty estimation on two
challenging computer vision tasks: horizon line detection and stereo disparity
estimation.
