  Variational Inference (VI) is an attractive alternative to Markov Chain Monte
Carlo (MCMC) due to its computational efficiency in the case of large datasets
and/or complex models with high-dimensional parameters. However, evaluating the
accuracy of variational approximations remains a challenge. Existing methods
characterize the quality of the whole variational distribution, which is almost
always poor in realistic applications, even if specific posterior functionals
such as the component-wise means or variances are accurate. Hence, these
diagnostics are of practical value only in limited circumstances. To address
this issue, we propose the TArgeted Diagnostic for Distribution Approximation
Accuracy (TADDAA), which uses many short parallel MCMC chains to obtain lower
bounds on the error of each posterior functional of interest. We also develop a
reliability check for TADDAA to determine when the lower bounds should not be
trusted. Numerical experiments validate the practical utility and computational
efficiency of our approach on a range of synthetic distributions and real-data
examples, including sparse logistic regression and Bayesian neural network
models.
