  Distributional regression aims to estimate the full conditional distribution
of a target variable, given covariates. Popular methods include linear and
tree-ensemble based quantile regression. We propose a neural network-based
distributional regression methodology called `engression'. An engression model
is generative in the sense that we can sample from the fitted conditional
distribution and is also suitable for high-dimensional outcomes. Furthermore,
we find that modelling the conditional distribution on training data can
constrain the fitted function outside of the training support, which offers a
new perspective to the challenging extrapolation problem in nonlinear
regression. In particular, for `pre-additive noise' models, where noise is
added to the covariates before applying a nonlinear transformation, we show
that engression can successfully perform extrapolation under some assumptions
such as monotonicity, whereas traditional regression approaches such as
least-squares or quantile regression fall short under the same assumptions. Our
empirical results, from both simulated and real data, validate the
effectiveness of the engression method and indicate that the pre-additive noise
model is typically suitable for many real-world scenarios. The software
implementations of engression are available in both R and Python.
