  Learning the distribution of a continuous or categorical response variable
$\boldsymbol y$ given its covariates $\boldsymbol x$ is a fundamental problem
in statistics and machine learning. Deep neural network-based supervised
learning algorithms have made great progress in predicting the mean of
$\boldsymbol y$ given $\boldsymbol x$, but they are often criticized for their
ability to accurately capture the uncertainty of their predictions. In this
paper, we introduce classification and regression diffusion (CARD) models,
which combine a denoising diffusion-based conditional generative model and a
pre-trained conditional mean estimator, to accurately predict the distribution
of $\boldsymbol y$ given $\boldsymbol x$. We demonstrate the outstanding
ability of CARD in conditional distribution prediction with both toy examples
and real-world datasets, the experimental results on which show that CARD in
general outperforms state-of-the-art methods, including Bayesian neural
network-based ones that are designed for uncertainty estimation, especially
when the conditional distribution of $\boldsymbol y$ given $\boldsymbol x$ is
multi-modal. In addition, we utilize the stochastic nature of the generative
model outputs to obtain a finer granularity in model confidence assessment at
the instance level for classification tasks.
