  Reversing a diffusion process by learning its score forms the heart of
diffusion-based generative modeling and for estimating properties of scientific
systems. The diffusion processes that are tractable center on linear processes
with a Gaussian stationary distribution. This limits the kinds of models that
can be built to those that target a Gaussian prior or more generally limits the
kinds of problems that can be generically solved to those that have
conditionally linear score functions. In this work, we introduce a family of
tractable denoising score matching objectives, called local-DSM, built using
local increments of the diffusion process. We show how local-DSM melded with
Taylor expansions enables automated training and score estimation with
nonlinear diffusion processes. To demonstrate these ideas, we use automated-DSM
to train generative models using non-Gaussian priors on challenging low
dimensional distributions and the CIFAR10 image dataset. Additionally, we use
the automated-DSM to learn the scores for nonlinear processes studied in
statistical physics.
