  Combining the merits of both denoising diffusion probabilistic models and
gradient boosting, the diffusion boosting paradigm is introduced for tackling
supervised learning problems. We develop Diffusion Boosted Trees (DBT), which
can be viewed as both a new denoising diffusion generative model parameterized
by decision trees (one single tree for each diffusion timestep), and a new
boosting algorithm that combines the weak learners into a strong learner of
conditional distributions without making explicit parametric assumptions on
their density forms. We demonstrate through experiments the advantages of DBT
over deep neural network-based diffusion models as well as the competence of
DBT on real-world regression tasks, and present a business application (fraud
detection) of DBT for classification on tabular data with the ability of
learning to defer.
