  Multiple penalized least squares (MPLS) models are a flexible approach to
find adaptive least squares solutions required to be simultaneously sparse and
smooth. This is particularly important when addressing real-life inverse
problems where there is no ground truth available, such as electrophysiological
source imaging. In this work we formalize a modified Newton-Raphson (MNR)
algorithm to estimate general MPLS models and propose its extension to perform
efficient optimization over the active set of selected features (AMNR). This
algorithm can be used to minimize continuously differentiable objective
functions with multiple restrictions, including sign constraints. We show that
these algorithms provide solutions with acceptable reconstruction in simulated
scenarios that do not cope with model assumptions, and for low n/p ratios. We
then use both algorithms for estimating different electroencephalography (EEG)
inverse models with multiple penalties. We also show how the AMNR allows us to
estimate new models in the EEG inverse problem context, such as nonnegative
versions of Smooth Garrote and Smooth LASSO. Synthetic data were used for a
comparison between solutions obtained using the two algorithms proposed here
and the least angle regression (LARS) algorithm, according to well-known
quality measures. A visual event-related EEG from healthy young subjects and a
resting-state EEG study on the relationship between cognitive ageing and
walking speed decline in active elders, were used to illustrate the usefulness
of the proposed methods in the analysis of real experimental data.
