  Diffusion-based generative models (DBGMs) perturb data to a target noise
distribution and reverse this process to generate samples. The choice of
noising process, or inference diffusion process, affects both likelihoods and
sample quality. For example, extending the inference process with auxiliary
variables leads to improved sample quality. While there are many such
multivariate diffusions to explore, each new one requires significant
model-specific analysis, hindering rapid prototyping and evaluation. In this
work, we study Multivariate Diffusion Models (MDMs). For any number of
auxiliary variables, we provide a recipe for maximizing a lower-bound on the
MDMs likelihood without requiring any model-specific analysis. We then
demonstrate how to parameterize the diffusion for a specified target noise
distribution; these two points together enable optimizing the inference
diffusion process. Optimizing the diffusion expands easy experimentation from
just a few well-known processes to an automatic search over all linear
diffusions. To demonstrate these ideas, we introduce two new specific
diffusions as well as learn a diffusion process on the MNIST, CIFAR10, and
ImageNet32 datasets. We show learned MDMs match or surpass bits-per-dims (BPDs)
relative to fixed choices of diffusions for a given dataset and model
architecture.
