  Consider the standard nonparametric regression model and take as estimator
the penalized least squares function. In this article, we study the trade-off
between closeness to the true function and complexity penalization of the
estimator, where complexity is described by a seminorm on a class of functions.
First, we present an exponential concentration inequality revealing the
concentration behavior of the trade-off of the penalized least squares
estimator around a nonrandom quantity, where such quantity depends on the
problem under consideration. Then, under some conditions and for the proper
choice of the tuning parameter, we obtain bounds for this nonrandom quantity.
We illustrate our results with some examples that include the smoothing splines
estimator.
