  Among Bayesian methods, Monte-Carlo dropout provides principled tools for
evaluating the epistemic uncertainty of neural networks. Its popularity
recently led to seminal works that proposed activating the dropout layers only
during inference for evaluating uncertainty. This approach, which we call
dropout injection, provides clear benefits over its traditional counterpart
(which we call embedded dropout) since it allows one to obtain a post hoc
uncertainty measure for any existing network previously trained without
dropout, avoiding an additional, time-consuming training process.
Unfortunately, no previous work compared injected and embedded dropout;
therefore, we provide the first thorough investigation, focusing on regression
problems. The main contribution of our work is to provide guidelines on the
effective use of injected dropout so that it can be a practical alternative to
the current use of embedded dropout. In particular, we show that its
effectiveness strongly relies on a suitable scaling of the corresponding
uncertainty measure, and we discuss the trade-off between negative
log-likelihood and calibration error as a function of the scale factor.
Experimental results on UCI data sets and crowd counting benchmarks support our
claim that dropout injection can effectively behave as a competitive post hoc
uncertainty quantification technique.
