  Flow-based generative models are powerful exact likelihood models with
efficient sampling and inference. Despite their computational efficiency,
flow-based models generally have much worse density modeling performance
compared to state-of-the-art autoregressive models. In this paper, we
investigate and improve upon three limiting design choices employed by
flow-based models in prior work: the use of uniform noise for dequantization,
the use of inexpressive affine flows, and the use of purely convolutional
conditioning networks in coupling layers. Based on our findings, we propose
Flow++, a new flow-based model that is now the state-of-the-art
non-autoregressive model for unconditional density estimation on standard image
benchmarks. Our work has begun to close the significant performance gap that
has so far existed between autoregressive models and flow-based models. Our
implementation is available at https://github.com/aravindsrinivas/flowpp
