  Score-based models have recently been introduced as a richer framework to
model distributions in high dimensions and are generally more suitable for
generative tasks. In score-based models, a generative task is formulated using
a parametric model (such as a neural network) to directly learn the gradient of
such high dimensional distributions, instead of the density functions
themselves, as is done traditionally. From the mathematical point of view, such
gradient information can be utilized in reverse by stochastic sampling to
generate diverse samples. However, from a computational perspective, existing
score-based models can be efficiently trained only if the forward or the
corruption process can be computed in closed form. By using the relationship
between the process and layers in a feed-forward network, we derive a
backpropagation-based procedure which we call Intermediate Generator
Optimization to utilize intermediate iterates of the process with negligible
computational overhead. The main advantage of IGO is that it can be
incorporated into any standard autoencoder pipeline for the generative task. We
analyze the sample complexity properties of IGO to solve downstream tasks like
Generative PCA. We show applications of the IGO on two dense predictive tasks
viz., image extrapolation, and point cloud denoising. Our experiments indicate
that obtaining an ensemble of generators for various time points is possible
using first-order methods.
