  Since the advent of the horseshoe priors for regularization, global-local
shrinkage methods have proved to be a fertile ground for the development of
Bayesian methodology in machine learning, specifically for high-dimensional
regression and classification problems. They have achieved remarkable success
in computation, and enjoy strong theoretical support. Most of the existing
literature has focused on the linear Gaussian case; see Bhadra et al. (2019b)
for a systematic survey. The purpose of the current article is to demonstrate
that the horseshoe regularization is useful far more broadly, by reviewing both
methodological and computational developments in complex models that are more
relevant to machine learning applications. Specifically, we focus on
methodological challenges in horseshoe regularization in nonlinear and
non-Gaussian models; multivariate models; and deep neural networks. We also
outline the recent computational developments in horseshoe shrinkage for
complex models along with a list of available software implementations that
allows one to venture out beyond the comfort zone of the canonical linear
regression problems.
