  Learning with Reproducing Kernel Hilbert Spaces (RKHS) has been widely used
in many scientific disciplines. Because a RKHS can be very flexible, it is
common to impose a regularization term in the optimization to prevent
overfitting. Standard RKHS learning employs the squared norm penalty of the
learning function. Despite its success, many challenges remain. In particular,
one cannot directly use the squared norm penalty for variable selection or data
extraction. Therefore, when there exists noise predictors, or the underlying
function has a sparse representation in the dual space, the performance of
standard RKHS learning can be suboptimal. In the literature,work has been
proposed on how to perform variable selection in RKHS learning, and a data
sparsity constraint was considered for data extraction. However, how to learn
in a RKHS with both variable selection and data extraction simultaneously
remains unclear. In this paper, we propose a unified RKHS learning method,
namely, DOuble Sparsity Kernel (DOSK) learning, to overcome this challenge. An
efficient algorithm is provided to solve the corresponding optimization
problem. We prove that under certain conditions, our new method can
asymptotically achieve variable selection consistency. Simulated and real data
results demonstrate that DOSK is highly competitive among existing approaches
for RKHS learning.
