  The tree-based ensembles are known for their outstanding performance in
classification and regression problems characterized by feature vectors
represented by mixed-type variables from various ranges and domains. However,
considering regression problems, they are primarily designed to provide
deterministic responses or model the uncertainty of the output with Gaussian or
parametric distribution. In this work, we introduce TreeFlow, the tree-based
approach that combines the benefits of using tree ensembles with the
capabilities of modeling flexible probability distributions using normalizing
flows. The main idea of the solution is to use a tree-based model as a feature
extractor and combine it with a conditional variant of normalizing flow.
Consequently, our approach is capable of modeling complex distributions for the
regression outputs. We evaluate the proposed method on challenging regression
benchmarks with varying volume, feature characteristics, and target
dimensionality. We obtain the SOTA results for both probabilistic and
deterministic metrics on datasets with multi-modal target distributions and
competitive results on unimodal ones compared to tree-based regression
baselines.
