  Diffusion probabilistic models (DPMs) have achieved impressive success in
high-resolution image synthesis, especially in recent large-scale text-to-image
generation applications. An essential technique for improving the sample
quality of DPMs is guided sampling, which usually needs a large guidance scale
to obtain the best sample quality. The commonly-used fast sampler for guided
sampling is DDIM, a first-order diffusion ODE solver that generally needs 100
to 250 steps for high-quality samples. Although recent works propose dedicated
high-order solvers and achieve a further speedup for sampling without guidance,
their effectiveness for guided sampling has not been well-tested before. In
this work, we demonstrate that previous high-order fast samplers suffer from
instability issues, and they even become slower than DDIM when the guidance
scale grows large. To further speed up guided sampling, we propose
DPM-Solver++, a high-order solver for the guided sampling of DPMs. DPM-Solver++
solves the diffusion ODE with the data prediction model and adopts thresholding
methods to keep the solution matches training data distribution. We further
propose a multistep variant of DPM-Solver++ to address the instability issue by
reducing the effective step size. Experiments show that DPM-Solver++ can
generate high-quality samples within only 15 to 20 steps for guided sampling by
pixel-space and latent-space DPMs.
