  Uncertainty estimation (UE) techniques -- such as the Gaussian process (GP),
Bayesian neural networks (BNN), Monte Carlo dropout (MCDropout) -- aim to
improve the interpretability of machine learning models by assigning an
estimated uncertainty value to each of their prediction outputs. However, since
too high uncertainty estimates can have fatal consequences in practice, this
paper analyzes the above techniques.
  Firstly, we show that GP methods always yield high uncertainty estimates on
out of distribution (OOD) data. Secondly, we show on a 2D toy example that both
BNNs and MCDropout do not give high uncertainty estimates on OOD samples.
Finally, we show empirically that this pitfall of BNNs and MCDropout holds on
real world datasets as well. Our insights (i) raise awareness for the more
cautious use of currently popular UE methods in Deep Learning, (ii) encourage
the development of UE methods that approximate GP-based methods -- instead of
BNNs and MCDropout, and (iii) our empirical setups can be used for verifying
the OOD performances of any other UE method. The source code is available at
https://github.com/epfml/uncertainity-estimation.
