  Normalizing flows and autoregressive models have been successfully combined
to produce state-of-the-art results in density estimation, via Masked
Autoregressive Flows (MAF), and to accelerate state-of-the-art WaveNet-based
speech synthesis to 20x faster than real-time, via Inverse Autoregressive Flows
(IAF). We unify and generalize these approaches, replacing the (conditionally)
affine univariate transformations of MAF/IAF with a more general class of
invertible univariate transformations expressed as monotonic neural networks.
We demonstrate that the proposed neural autoregressive flows (NAF) are
universal approximators for continuous probability distributions, and their
greater expressivity allows them to better capture multimodal target
distributions. Experimentally, NAF yields state-of-the-art performance on a
suite of density estimation tasks and outperforms IAF in variational
autoencoders trained on binarized MNIST.
