  The SparseStep algorithm is presented for the estimation of a sparse
parameter vector in the linear regression problem. The algorithm works by
adding an approximation of the exact counting norm as a constraint on the model
parameters and iteratively strengthening this approximation to arrive at a
sparse solution. Theoretical analysis of the penalty function shows that the
estimator yields unbiased estimates of the parameter vector. An iterative
majorization algorithm is derived which has a straightforward implementation
reminiscent of ridge regression. In addition, the SparseStep algorithm is
compared with similar methods through a rigorous simulation study which shows
it often outperforms existing methods in both model fit and prediction
accuracy.
