  Denoising diffusion (score-based) generative models have recently achieved
significant accomplishments in generating realistic and diverse data. These
approaches define a forward diffusion process for transforming data into noise
and a backward denoising process for sampling data from noise. Unfortunately,
the generation process of current denoising diffusion models is notoriously
slow due to the lengthy iterative noise estimations, which rely on cumbersome
neural networks. It prevents the diffusion models from being widely deployed,
especially on edge devices. Previous works accelerate the generation process of
diffusion model (DM) via finding shorter yet effective sampling trajectories.
However, they overlook the cost of noise estimation with a heavy network in
every iteration. In this work, we accelerate generation from the perspective of
compressing the noise estimation network. Due to the difficulty of retraining
DMs, we exclude mainstream training-aware compression paradigms and introduce
post-training quantization (PTQ) into DM acceleration. However, the output
distributions of noise estimation networks change with time-step, making
previous PTQ methods fail in DMs since they are designed for single-time step
scenarios. To devise a DM-specific PTQ method, we explore PTQ on DM in three
aspects: quantized operations, calibration dataset, and calibration metric. We
summarize and use several observations derived from all-inclusive
investigations to formulate our method, which especially targets the unique
multi-time-step structure of DMs. Experimentally, our method can directly
quantize full-precision DMs into 8-bit models while maintaining or even
improving their performance in a training-free manner. Importantly, our method
can serve as a plug-and-play module on other fast-sampling methods, e.g., DDIM.
The code is available at https://github.com/42Shawn/PTQ4DM .
