  Diffusion models have emerged as a formidable tool for training-free
conditional generation.However, a key hurdle in inference-time guidance
techniques is the need for compute-heavy backpropagation through the diffusion
network for estimating the guidance direction. Moreover, these techniques often
require handcrafted parameter tuning on a case-by-case basis. Although some
recent works have introduced minimal compute methods for linear inverse
problems, a generic lightweight guidance solution to both linear and non-linear
guidance problems is still missing. To this end, we propose Dreamguider, a
method that enables inference-time guidance without compute-heavy
backpropagation through the diffusion network. The key idea is to regulate the
gradient flow through a time-varying factor. Moreover, we propose an empirical
guidance scale that works for a wide variety of tasks, hence removing the need
for handcrafted parameter tuning. We further introduce an effective lightweight
augmentation strategy that significantly boosts the performance during
inference-time guidance. We present experiments using Dreamguider on multiple
tasks across multiple datasets and models to show the effectiveness of the
proposed modules. To facilitate further research, we will make the code public
after the review process.
