  The problem of detecting the Out-of-Distribution (OoD) inputs is of paramount
importance for Deep Neural Networks. It has been previously shown that even
Deep Generative Models that allow estimating the density of the inputs may not
be reliable and often tend to make over-confident predictions for OoDs,
assigning to them a higher density than to the in-distribution data. This
over-confidence in a single model can be potentially mitigated with Bayesian
inference over the model parameters that take into account epistemic
uncertainty. This paper investigates three approaches to Bayesian inference:
stochastic gradient Markov chain Monte Carlo, Bayes by Backpropagation, and
Stochastic Weight Averaging-Gaussian. The inference is implemented over the
weights of the deep neural networks that parameterize the likelihood of the
Variational Autoencoder. We empirically evaluate the approaches against several
benchmarks that are often used for OoD detection: estimation of the marginal
likelihood utilizing sampled model ensemble, typicality test, disagreement
score, and Watanabe-Akaike Information Criterion. Finally, we introduce two
simple scores that demonstrate the state-of-the-art performance.
