  Flow based models such as Real NVP are an extremely powerful approach to
density estimation. However, existing flow based models are restricted to
transforming continuous densities over a continuous input space into similarly
continuous distributions over continuous latent variables. This makes them
poorly suited for modeling and representing discrete structures in data
distributions, for example class membership or discrete symmetries. To address
this difficulty, we present a normalizing flow architecture which relies on
domain partitioning using locally invertible functions, and possesses both real
and discrete valued latent variables. This Real and Discrete (RAD) approach
retains the desirable normalizing flow properties of exact sampling, exact
inference, and analytically computable probabilities, while at the same time
allowing simultaneous modeling of both continuous and discrete structure in a
data distribution.
