  Diffusion models show promising generation capability for a variety of data.
Despite their high generation quality, the inference for diffusion models is
still time-consuming due to the numerous sampling iterations required. To
accelerate the inference, we propose ReDi, a simple yet learning-free
Retrieval-based Diffusion sampling framework. From a precomputed knowledge
base, ReDi retrieves a trajectory similar to the partially generated trajectory
at an early stage of generation, skips a large portion of intermediate steps,
and continues sampling from a later step in the retrieved trajectory. We
theoretically prove that the generation performance of ReDi is guaranteed. Our
experiments demonstrate that ReDi improves the model inference efficiency by 2x
speedup. Furthermore, ReDi is able to generalize well in zero-shot cross-domain
image generation such as image stylization.
