  Well-calibrated probabilistic regression models are a crucial learning
component in robotics applications as datasets grow rapidly and tasks become
more complex. Unfortunately, classical regression models are usually either
probabilistic kernel machines with a flexible structure that does not scale
gracefully with data or deterministic and vastly scalable automata, albeit with
a restrictive parametric form and poor regularization. In this paper, we
consider a probabilistic hierarchical modeling paradigm that combines the
benefits of both worlds to deliver computationally efficient representations
with inherent complexity regularization. The presented approaches are
probabilistic interpretations of local regression techniques that approximate
nonlinear functions through a set of local linear or polynomial units.
Importantly, we rely on principles from Bayesian nonparametrics to formulate
flexible models that adapt their complexity to the data and can potentially
encompass an infinite number of components. We derive two efficient variational
inference techniques to learn these representations and highlight the
advantages of hierarchical infinite local regression models, such as dealing
with non-smooth functions, mitigating catastrophic forgetting, and enabling
parameter sharing and fast predictions. Finally, we validate this approach on
large inverse dynamics datasets and test the learned models in real-world
control scenarios.
