  In statistics, the least absolute shrinkage and selection operator (Lasso) is
a regression method that performs both variable selection and regularization.
There is a lot of literature available, discussing the statistical properties
of the regression coefficients estimated by the Lasso method. However, there
lacks a comprehensive review discussing the algorithms to solve the
optimization problem in Lasso. In this review, we summarize five representative
algorithms to optimize the objective function in Lasso, including the iterative
shrinkage threshold algorithm (ISTA), fast iterative shrinkage-thresholding
algorithms (FISTA), coordinate gradient descent algorithm (CGDA), smooth L1
algorithm (SLA), and path following algorithm (PFA). Additionally, we also
compare their convergence rate, as well as their potential strengths and
weakness.
