  We investigate $L_2$ boosting in the context of kernel regression. Kernel
smoothers, in general, lack appealing traits like symmetry and positive
definiteness, which are critical not only for understanding theoretical aspects
but also for achieving good practical performance. We consider a
projection-based smoother (Huang and Chen, 2008) that is symmetric, positive
definite, and shrinking. Theoretical results based on the orthonormal
decomposition of the smoother reveal additional insights into the boosting
algorithm. In our asymptotic framework, we may replace the full-rank smoother
with a low-rank approximation. We demonstrate that the smoother's low-rank
($d(n)$) is bounded above by $O(h^{-1})$, where $h$ is the bandwidth. Our
numerical findings show that, in terms of prediction accuracy, low-rank
smoothers may outperform full-rank smoothers. Furthermore, we show that the
boosting estimator with low-rank smoother achieves the optimal convergence
rate. Finally, to improve the performance of the boosting algorithm in the
presence of outliers, we propose a novel robustified boosting algorithm which
can be used with any smoother discussed in the study. We investigate the
numerical performance of the proposed approaches using simulations and a
real-world case.
