  With the rise of the popularity and usage of neural networks, trustworthy
uncertainty estimation is becoming increasingly essential. One of the most
prominent uncertainty estimation methods is Deep Ensembles (Lakshminarayanan et
al., 2017) . A classical parametric model has uncertainty in the parameters due
to the fact that the data on which the model is build is a random sample. A
modern neural network has an additional uncertainty component since the
optimization of the network is random. Lakshminarayanan et al. (2017) noted
that Deep Ensembles do not incorporate the classical uncertainty induced by the
effect of finite data. In this paper, we present a computationally cheap
extension of Deep Ensembles for the regression setting, called Bootstrapped
Deep Ensembles, that explicitly takes this classical effect of finite data into
account using a modified version of the parametric bootstrap. We demonstrate
through an experimental study that our method significantly improves upon
standard Deep Ensembles
