  In high-dimensional linear regression, the goal pursued here is to estimate
an unknown regression function using linear combinations of a suitable set of
covariates. One of the key assumptions for the success of any statistical
procedure in this setup is to assume that the linear combination is sparse in
some sense, for example, that it involves only few covariates. We consider a
general, non necessarily linear, regression with Gaussian noise and study a
related question that is to find a linear combination of approximating
functions, which is at the same time sparse and has small mean squared error
(MSE). We introduce a new estimation procedure, called Exponential Screening
that shows remarkable adaptation properties. It adapts to the linear
combination that optimally balances MSE and sparsity, whether the latter is
measured in terms of the number of non-zero entries in the combination
($\ell_0$ norm) or in terms of the global weight of the combination ($\ell_1$
norm). The power of this adaptation result is illustrated by showing that
Exponential Screening solves optimally and simultaneously all the problems of
aggregation in Gaussian regression that have been discussed in the literature.
Moreover, we show that the performance of the Exponential Screening estimator
cannot be improved in a minimax sense, even if the optimal sparsity is known in
advance. The theoretical and numerical superiority of Exponential Screening
compared to state-of-the-art sparse procedures is also discussed.
