  Penalized estimation can conduct variable selection and parameter estimation
simultaneously. The general framework is to minimize a loss function subject to
a penalty designed to generate sparse variable selection. The
majorization-minimization (MM) algorithm is a computational scheme for
stability and simplicity, and the MM algorithm has been widely applied in
penalized estimation. Much of the previous work have focused on convex loss
functions such as generalized linear models. When data are contaminated with
outliers, robust loss functions can generate more reliable estimates. Recent
literature has witnessed a growing impact of nonconvex loss-based methods,
which can generate robust estimation for data contaminated with outliers. This
article investigates MM algorithm for penalized estimation, provide innovative
optimality conditions and establish convergence theory with both convex and
nonconvex loss functions. With respect to applications, we focus on several
nonconvex loss functions, which were formerly studied in machine learning for
regression and classification problems. Performance of the proposed algorithms
are evaluated on simulated and real data including healthcare costs and cancer
clinical status. Efficient implementations of the algorithms are available in
the R package mpath in CRAN.
