  Diffusion models (DMs) demonstrate potent image generation capabilities in
various generative modeling tasks. Nevertheless, their primary limitation lies
in slow sampling speed, requiring hundreds or thousands of sequential function
evaluations through large neural networks to generate high-quality images.
Sampling from DMs can be seen alternatively as solving corresponding stochastic
differential equations (SDEs) or ordinary differential equations (ODEs). In
this work, we formulate the sampling process as an extended reverse-time SDE
(ER SDE), unifying prior explorations into ODEs and SDEs. Leveraging the
semi-linear structure of ER SDE solutions, we offer exact solutions and
arbitrarily high-order approximate solutions for VP SDE and VE SDE,
respectively. Based on the solution space of the ER SDE, we yield mathematical
insights elucidating the superior performance of ODE solvers over SDE solvers
in terms of fast sampling. Additionally, we unveil that VP SDE solvers stand on
par with their VE SDE counterparts. Finally, we devise fast and training-free
samplers, ER-SDE-Solvers, achieving state-of-the-art performance across all
stochastic samplers. Experimental results demonstrate achieving 3.45 FID in 20
function evaluations and 2.24 FID in 50 function evaluations on the ImageNet
$64\times64$ dataset.
