  Precision matrix estimation in a multivariate Gaussian model is fundamental
to network estimation. Although there exist both Bayesian and frequentist
approaches to this, it is difficult to obtain good Bayesian and frequentist
properties under the same prior--penalty dual. To bridge this gap, our
contribution is a novel prior--penalty dual that closely approximates the
graphical horseshoe prior and penalty, and performs well in both Bayesian and
frequentist senses. A chief difficulty with the horseshoe prior is a lack of
closed form expression of the density function, which we overcome in this
article. In terms of theory, we establish posterior convergence rate of the
precision matrix that matches the oracle rate, in addition to the frequentist
consistency of the MAP estimator. In addition, our results also provide
theoretical justifications for previously developed approaches that have been
unexplored so far, e.g. for the graphical horseshoe prior. Computationally
efficient EM and MCMC algorithms are developed respectively for the penalized
likelihood and fully Bayesian estimation problems. In numerical experiments,
the horseshoe-based approaches echo their superior theoretical properties by
comprehensively outperforming the competing methods. A protein--protein
interaction network estimation in B-cell lymphoma is considered to validate the
proposed methodology.
