  This paper develops a general theory on rates of convergence of penalized
spline estimators for function estimation when the likelihood functional is
concave in candidate functions, where the likelihood is interpreted in a broad
sense that includes conditional likelihood, quasi-likelihood, and
pseudo-likelihood. The theory allows all feasible combinations of the spline
degree, the penalty order, and the smoothness of the unknown functions.
According to this theory, the asymptotic behaviors of the penalized spline
estimators depends on interplay between the spline knot number and the penalty
parameter. The general theory is applied to obtain results in a variety of
contexts, including regression, generalized regression such as logistic
regression and Poisson regression, density estimation, conditional hazard
function estimation for censored data, quantile regression, diffusion function
estimation for a diffusion type process, and estimation of spectral density
function of a stationary time series. For multi-dimensional function
estimation, the theory (presented in the Supplementary Material) covers both
penalized tensor product splines and penalized bivariate splines on
triangulations.
