  Over-parameterized models like deep nets and random forests have become very
popular in machine learning. However, the natural goals of continuity and
differentiability, common in regression models, are now often ignored in modern
overparametrized, locally-adaptive models. We propose a general framework to
construct a global continuous and differentiable model based on a weighted
average of locally learned models in corresponding local regions. This model is
competitive in dealing with data with different densities or scales of function
values in different local regions. We demonstrate that when we mix kernel ridge
and polynomial regression terms in the local models, and stitch them together
continuously, we achieve faster statistical convergence in theory and improved
performance in various practical settings.
