  This paper considers the penalized least squares estimator with arbitrary
convex penalty. When the observation noise is Gaussian, we show that the
prediction error is a subgaussian random variable concentrated around its
median. We apply this concentration property to derive sharp oracle
inequalities for the prediction error of the LASSO, the group LASSO and the
SLOPE estimators, both in probability and in expectation. In contrast to the
previous work on the LASSO type methods, our oracle inequalities in probability
are obtained at any confidence level for estimators with tuning parameters that
do not depend on the confidence level. This is also the reason why we are able
to establish sparsity oracle bounds in expectation for the LASSO type
estimators, while the previously known techniques did not allow for the control
of the expected risk. In addition, we show that the concentration rate in the
oracle inequalities is better than it was commonly understood before.
