  We investigate the credible sets and marginal credible intervals resulting
from the horseshoe prior in the sparse multivariate normal means model. We do
so in an adaptive setting without assuming knowledge of the sparsity level
(number of signals). We consider both the hierarchical Bayes method of putting
a prior on the unknown sparsity level and the empirical Bayes method with the
sparsity level estimated by maximum marginal likelihood. We show that credible
balls and marginal credible intervals have good frequentist coverage and
optimal size if the sparsity level of the prior is set correctly. By general
theory honest confidence sets cannot adapt in size to an unknown sparsity
level. Accordingly the hierarchical and empirical Bayes credible sets based on
the horseshoe prior are not honest over the full parameter space. We show that
this is due to over-shrinkage for certain parameters and characterise the set
of parameters for which credible balls and marginal credible intervals do give
correct uncertainty quantification. In particular we show that the fraction of
false discoveries by the marginal Bayesian procedure is controlled by a correct
choice of cut-off.
