  The goal of this paper is to contrast and survey the major advances in two of
the most commonly used high-dimensional techniques, namely, the Lasso and
horseshoe regularization. Lasso is a gold standard for predictor selection
while horseshoe is a state-of-the-art Bayesian estimator for sparse signals.
Lasso is fast and scalable and uses convex optimization whilst the horseshoe is
non-convex. Our novel perspective focuses on three aspects: (i) theoretical
optimality in high dimensional inference for the Gaussian sparse model and
beyond, (ii) efficiency and scalability of computation and (iii) methodological
development and performance.
