  In classification applications, we often want probabilistic predictions to
reflect confidence or uncertainty. Dropout, a commonly used training technique,
has recently been linked to Bayesian inference, yielding an efficient way to
quantify uncertainty in neural network models. However, as previously
demonstrated, confidence estimates computed with a naive implementation of
dropout can be poorly calibrated, particularly when using convolutional
networks. In this paper, through the lens of ensemble learning, we associate
calibration error with the correlation between the models sampled with dropout.
Motivated by this, we explore the use of structured dropout to promote model
diversity and improve confidence calibration. We use the SVHN, CIFAR-10 and
CIFAR-100 datasets to empirically compare model diversity and confidence errors
obtained using various dropout techniques. We also show the merit of structured
dropout in a Bayesian active learning application.
