  Diffusion models have demonstrated empirical successes in various
applications and can be adapted to task-specific needs via guidance. This paper
introduces a form of gradient guidance for adapting or fine-tuning diffusion
models towards user-specified optimization objectives. We study the theoretic
aspects of a guided score-based sampling process, linking the gradient-guided
diffusion model to first-order optimization. We show that adding gradient
guidance to the sampling process of a pre-trained diffusion model is
essentially equivalent to solving a regularized optimization problem, where the
regularization term acts as a prior determined by the pre-training data.
Diffusion models are able to learn data's latent subspace, however, explicitly
adding the gradient of an external objective function to the sample process
would jeopardize the structure in generated samples. To remedy this issue, we
consider a modified form of gradient guidance based on a forward prediction
loss, which leverages the pre-trained score function to preserve the latent
structure in generated samples. We further consider an iteratively fine-tuned
version of gradient-guided diffusion where one can query gradients at newly
generated data points and update the score network using new samples. This
process mimics a first-order optimization iteration in expectation, for which
we proved O(1/K) convergence rate to the global optimum when the objective
function is concave.
