  Text-to-image diffusion models have demonstrated unprecedented capabilities
for flexible and realistic image synthesis. Nevertheless, these models rely on
a time-consuming sampling procedure, which has motivated attempts to reduce
their latency. When improving efficiency, researchers often use the original
diffusion model to train an additional network designed specifically for fast
image generation. In contrast, our approach seeks to reduce latency directly,
without any retraining, fine-tuning, or knowledge distillation. In particular,
we find the repeated calculation of attention maps to be costly yet redundant,
and instead suggest reusing them during sampling. Our specific reuse strategies
are based on ODE theory, which implies that the later a map is reused, the
smaller the distortion in the final image. We empirically compare these reuse
strategies with few-step sampling procedures of comparable latency, finding
that reuse generates images that are closer to those produced by the original
high-latency diffusion model.
