  In this paper,we consider a high-dimensional statistical estimation problem
in which the the number of parameters is comparable or larger than the sample
size. We present a unified analysis of the performance guarantees of
exponential weighted aggregation and penalized estimators with a general class
of data losses and priors which encourage objects which conform to some notion
of simplicity/complexity. More precisely, we show that these two estimators
satisfy sharp oracle inequalities for prediction ensuring their good
theoretical performances. We also highlight the differences between them. When
the noise is random, we provide oracle inequalities in probability using
concentration inequalities. These results are then applied to several instances
including the Lasso, the group Lasso, their analysis-type counterparts, the
$\ell_\infty$ and the nuclear norm penalties. All our estimators can be
efficiently implemented using proximal splitting algorithms.
