  Bayesian predictive inference provides a coherent description of entire
predictive uncertainty through predictive distributions. We examine several
widely used sparsity priors from the predictive (as opposed to estimation)
inference viewpoint. To start, we investigate predictive distributions in the
context of a high-dimensional Gaussian observation with a known variance but an
unknown sparse mean under the Kullback-Leibler loss. First, we show that LASSO
(Laplace) priors are incapable of achieving rate-optimal predictive
distributions. However, deploying the Laplace prior inside the Spike-and-Slab
framework (e.g. with the Spike-and-Slab LASSO prior), rate-minimax performance
can be attained with properly tuned parameters (depending on the sparsity level
sn). We highlight the discrepancy between prior calibration for the purpose of
prediction and estimation. Going further, we investigate popular hierarchical
priors which are known to attain adaptive rate-minimax performance for
estimation. Whether or not they are rate-minimax also for predictive inference
has, until now, been unclear. We answer affirmatively by showing that
hierarchical Spike-and-Slab priors are adaptive and attain the minimax rate
without the knowledge of sn. This is the first rate-adaptive result in the
literature on predictive density estimation in sparse setups. Building on the
sparse normal-means model, we extend our adaptive rate findings to the case of
sparse high-dimensional regression with Spike-and-Slab priors. All of these
results underscore benefits of fully Bayesian predictive inference.
