  Generative flows are promising tractable models for density modeling that
define probabilistic distributions with invertible transformations. However,
tractability imposes architectural constraints on generative flows, making them
less expressive than other types of generative models. In this work, we study a
previously overlooked constraint that all the intermediate representations must
have the same dimensionality with the original data due to invertibility,
limiting the width of the network. We tackle this constraint by augmenting the
data with some extra dimensions and jointly learning a generative flow for
augmented data as well as the distribution of augmented dimensions under a
variational inference framework. Our approach, VFlow, is a generalization of
generative flows and therefore always performs better. Combining with existing
generative flows, VFlow achieves a new state-of-the-art 2.98 bits per dimension
on the CIFAR-10 dataset and is more compact than previous models to reach
similar modeling quality.
