  Gradient boosting is a sequential ensemble method that fits a new base
learner to the gradient of the remaining loss at each step. We propose a novel
family of gradient boosting, Wasserstein gradient boosting, which fits a new
base learner to an exactly or approximately available Wasserstein gradient of a
loss functional on the space of probability distributions. Wasserstein gradient
boosting returns a set of particles that approximates a target probability
distribution assigned at each input. In probabilistic prediction, a parametric
probability distribution is often specified on the space of output variables,
and a point estimate of the output-distribution parameter is produced for each
input by a model. Our main application of Wasserstein gradient boosting is a
novel distributional estimate of the output-distribution parameter, which
approximates the posterior distribution over the output-distribution parameter
determined pointwise at each data point. We empirically demonstrate the
superior performance of the probabilistic prediction by Wasserstein gradient
boosting in comparison with various existing methods.
