  In this work we are interested in the problems of supervised learning and
variable selection when the input-output dependence is described by a nonlinear
function depending on a few variables. Our goal is to consider a sparse
nonparametric model, hence avoiding linear or additive models. The key idea is
to measure the importance of each variable in the model by making use of
partial derivatives. Based on this intuition we propose a new notion of
nonparametric sparsity and a corresponding least squares regularization scheme.
Using concepts and results from the theory of reproducing kernel Hilbert spaces
and proximal methods, we show that the proposed learning algorithm corresponds
to a minimization problem which can be provably solved by an iterative
procedure. The consistency properties of the obtained estimator are studied
both in terms of prediction and selection performance. An extensive empirical
analysis shows that the proposed method performs favorably with respect to the
state-of-the-art methods.
