  We propose a new way of constructing invertible neural networks by combining
simple building blocks with a novel set of composition rules. This leads to a
rich set of invertible architectures, including those similar to ResNets.
Inversion is achieved with a locally convergent iterative procedure that is
parallelizable and very fast in practice. Additionally, the determinant of the
Jacobian can be computed analytically and efficiently, enabling their
generative use as flow models. To demonstrate their flexibility, we show that
our invertible neural networks are competitive with ResNets on MNIST and
CIFAR-10 classification. When trained as generative models, our invertible
networks achieve competitive likelihoods on MNIST, CIFAR-10 and ImageNet 32x32,
with bits per dimension of 0.98, 3.32 and 4.06 respectively.
