  We introduce Deep Sigma Point Processes, a class of parametric models
inspired by the compositional structure of Deep Gaussian Processes (DGPs). Deep
Sigma Point Processes (DSPPs) retain many of the attractive features of
(variational) DGPs, including mini-batch training and predictive uncertainty
that is controlled by kernel basis functions. Importantly, since DSPPs admit a
simple maximum likelihood inference procedure, the resulting predictive
distributions are not degraded by any posterior approximations. In an extensive
empirical comparison on univariate and multivariate regression tasks we find
that the resulting predictive distributions are significantly better calibrated
than those obtained with other probabilistic methods for scalable regression,
including variational DGPs--often by as much as a nat per datapoint.
