  We propose a fast method for solving compressed sensing, Lasso regression,
and Logistic Lasso regression problems that iteratively runs an appropriate
solver using an active set approach. We design a strategy to update the active
set that achieves a large speedup over a single call of several solvers,
including gradient projection for sparse reconstruction (GPSR), lassoglm of
Matlab, and glmnet. For compressed sensing, the hybrid of our method and GPSR
is 31.41 times faster than GPSR on average for Gaussian ensembles and 25.64
faster on average for binary ensembles. For Lasso regression, the hybrid of our
method and GPSR achieves a 30.67-fold average speedup in our experiments. In
our experiments on Logistic Lasso regression, the hybrid of our method and
lassoglm gives an 11.95-fold average speedup, and the hybrid of our method and
glmnet gives a 1.40-fold average speedup.
