  Predictive posterior densities (PPDs) are of interest in approximate Bayesian
inference. Typically, these are estimated by simple Monte Carlo (MC) averages
using samples from the approximate posterior. We observe that the
signal-to-noise ratio (SNR) of such estimators can be extremely low. An
analysis for exact inference reveals SNR decays exponentially as there is an
increase in (a) the mismatch between training and test data, (b) the
dimensionality of the latent space, or (c) the size of the test data relative
to the training data. Further analysis extends these results to approximate
inference. To remedy the low SNR problem, we propose replacing simple MC
sampling with importance sampling using a proposal distribution optimized at
test time on a variational proxy for the SNR and demonstrate that this yields
greatly improved estimates.
