  This paper investigates the high-dimensional linear regression with highly
correlated covariates. In this setup, the traditional sparsity assumption on
the regression coefficients often fails to hold, and consequently many model
selection procedures do not work. To address this challenge, we model the
variations of covariates by a factor structure. Specifically, strong
correlations among covariates are explained by common factors and the remaining
variations are interpreted as idiosyncratic components of each covariate. This
leads to a factor-adjusted regression model with both common factors and
idiosyncratic components as covariates. We generalize the traditional sparsity
assumption accordingly and assume that all common factors but only a small
number of idiosyncratic components contribute to the response. A Bayesian
procedure with a spike-and-slab prior is then proposed for parameter estimation
and model selection. Simulation studies show that our Bayesian method
outperforms its lasso analogue, manifests insensitivity to the overestimates of
the number of common factors, pays a negligible price in the no correlation
case, and scales up well with increasing sample size, dimensionality and
sparsity. Numerical results on a real dataset of U.S. bond risk premia and
macroeconomic indicators lend strong support to our methodology.
