  Probabilistic Regression refers to predicting a full probability density
function for the target conditional on the features. We present a nonparametric
approach to this problem which combines base classifiers (typically gradient
boosted forests) trained on different coarsenings of the target value. By
combining such classifiers and averaging the resulting densities, we are able
to compute precise conditional densities with minimal assumptions on the shape
or form of the density. We combine this approach with a structured
cross-entropy loss function which serves to regularize and smooth the resulting
densities. Prediction intervals computed from these densities are shown to have
high fidelity in practice. Furthermore, examining the properties of these
densities on particular observations can provide valuable insight. We
demonstrate this approach on a variety of datasets and show competitive
performance, particularly on larger datasets.
