  Diffusion models learn to denoise data and the trained denoiser is then used
to generate new samples from the data distribution. In this paper, we revisit
the diffusion sampling process and identify a fundamental cause of sample
quality degradation: the denoiser is poorly estimated in regions that are far
Outside Of the training Distribution (OOD), and the sampling process inevitably
evaluates in these OOD regions. This can become problematic for all sampling
methods, especially when we move to parallel sampling which requires us to
initialize and update the entire sample trajectory of dynamics in parallel,
leading to many OOD evaluations. To address this problem, we introduce a new
self-supervised training objective that differentiates the levels of noise
added to a sample, leading to improved OOD denoising performance. The approach
is based on our observation that diffusion models implicitly define a
log-likelihood ratio that distinguishes distributions with different amounts of
noise, and this expression depends on denoiser performance outside the standard
training distribution. We show by diverse experiments that the proposed
contrastive diffusion training is effective for both sequential and parallel
settings, and it improves the performance and speed of parallel samplers
significantly.
