  One of the main challenges of deep learning tools is their inability to
capture model uncertainty. While Bayesian deep learning can be used to tackle
the problem, Bayesian neural networks often require more time and computational
power to train than deterministic networks. Our work explores whether fully
Bayesian networks are needed to successfully capture model uncertainty. We vary
the number and position of Bayesian layers in a network and compare their
performance on active learning with the MNIST dataset. We found that we can
fully capture the model uncertainty by using only a few Bayesian layers near
the output of the network, combining the advantages of deterministic and
Bayesian networks.
