  We investigate the modeling and the numerical solution of machine learning
problems with prediction functions which are linear combinations of elements of
a possibly infinite-dimensional dictionary. We propose a novel flexible
composite regularization model, which makes it possible to incorporate various
priors on the coefficients of the prediction function, including sparsity and
hard constraints. We show that the estimators obtained by minimizing the
regularized empirical risk are consistent in a statistical sense, and we design
an error-tolerant composite proximal thresholding algorithm for computing such
estimators. New results on the asymptotic behavior of the proximal
forward-backward splitting method are derived and exploited to establish the
convergence properties of the proposed algorithm. In particular, our method
features a $o(1/m)$ convergence rate in objective values.
