  Most of the existing diffusion models use Gaussian noise for training and
sampling across all time steps, which may not optimally account for the
frequency contents reconstructed by the denoising network. Despite the diverse
applications of correlated noise in computer graphics, its potential for
improving the training process has been underexplored. In this paper, we
introduce a novel and general class of diffusion models taking correlated noise
within and across images into account. More specifically, we propose a
time-varying noise model to incorporate correlated noise into the training
process, as well as a method for fast generation of correlated noise mask. Our
model is built upon deterministic diffusion models and utilizes blue noise to
help improve the generation quality compared to using Gaussian white (random)
noise only. Further, our framework allows introducing correlation across images
within a single mini-batch to improve gradient flow. We perform both
qualitative and quantitative evaluations on a variety of datasets using our
method, achieving improvements on different tasks over existing deterministic
diffusion models in terms of FID metric.
