  Bayesian coresets speed up posterior inference in the large-scale data regime
by approximating the full-data log-likelihood function with a surrogate
log-likelihood based on a small, weighted subset of the data. But while
Bayesian coresets and methods for construction are applicable in a wide range
of models, existing theoretical analysis of the posterior inferential error
incurred by coreset approximations only apply in restrictive settings -- i.e.,
exponential family models, or models with strong log-concavity and smoothness
assumptions. This work presents general upper and lower bounds on the
Kullback-Leibler (KL) divergence of coreset approximations that reflect the
full range of applicability of Bayesian coresets. The lower bounds require only
mild model assumptions typical of Bayesian asymptotic analyses, while the upper
bounds require the log-likelihood functions to satisfy a generalized
subexponentiality criterion that is weaker than conditions used in earlier
work. The lower bounds are applied to obtain fundamental limitations on the
quality of coreset approximations, and to provide a theoretical explanation for
the previously-observed poor empirical performance of importance sampling-based
construction methods. The upper bounds are used to analyze the performance of
recent subsample-optimize methods. The flexibility of the theory is
demonstrated in validation experiments involving multimodal, unidentifiable,
heavy-tailed Bayesian posterior distributions.
