  Although Bayesian variable selection methods have been intensively studied,
their routine use in practice has not caught up with their non-Bayesian
counterparts such as Lasso, likely due to difficulties in both computations and
flexibilities of prior choices. To ease these challenges, we propose the
neuronized priors to unify and extend some popular shrinkage priors, such as
Laplace, Cauchy, horseshoe, and spike-and-slab priors. A neuronized prior can
be written as the product of a Gaussian weight variable and a scale variable
transformed from Gaussian via an activation function. Compared with classic
spike-and-slab priors, the neuronized priors achieve the same explicit variable
selection without employing any latent indicator variables, which results in
both more efficient and flexible posterior sampling and more effective
posterior modal estimation. Theoretically, we provide specific conditions on
the neuronized formulation to achieve the optimal posterior contraction rate,
and show that a broadly applicable MCMC algorithm achieves an exponentially
fast convergence rate under the neuronized formulation. We also examine various
simulated and real data examples and demonstrate that using the neuronization
representation is computationally more or comparably efficient than its
standard counterpart in all well-known cases. An R package NPrior is provided
in the CRAN for using neuronized priors in Bayesian linear regression.
