  Bayesian Inference offers principled tools to tackle many critical problems
with modern neural networks such as poor calibration and generalization, and
data inefficiency. However, scaling Bayesian inference to large architectures
is challenging and requires restrictive approximations. Monte Carlo Dropout has
been widely used as a relatively cheap way for approximate Inference and to
estimate uncertainty with deep neural networks. Traditionally, the dropout mask
is sampled independently from a fixed distribution. Recent works show that the
dropout mask can be viewed as a latent variable, which can be inferred with
variational inference. These methods face two important challenges: (a) the
posterior distribution over masks can be highly multi-modal which can be
difficult to approximate with standard variational inference and (b) it is not
trivial to fully utilize sample-dependent information and correlation among
dropout masks to improve posterior estimation. In this work, we propose
GFlowOut to address these issues. GFlowOut leverages the recently proposed
probabilistic framework of Generative Flow Networks (GFlowNets) to learn the
posterior distribution over dropout masks. We empirically demonstrate that
GFlowOut results in predictive distributions that generalize better to
out-of-distribution data, and provide uncertainty estimates which lead to
better performance in downstream tasks.
