  Rates of convergence for empirical risk minimizers have been well studied in
the literature. In this paper, we aim to provide a complementary set of
results, in particular by showing that after normalization, the risk of the
empirical minimizer concentrates on a single point. Such results have been
established by~\cite{chatterjee2014new} for constrained estimators in the
normal sequence model. We first generalize and sharpen this result to
regularized least squares with convex penalties, making use of a "direct"
argument based on Borell's theorem. We then study generalizations to other loss
functions, including the negative log-likelihood for exponential families
combined with a strictly convex regularization penalty. The results in this
general setting are based on more "indirect" arguments as well as on
concentration inequalities for maxima of empirical processes.
