  This paper introduces a novel generative model for discrete distributions
based on continuous normalizing flows on the submanifold of factorizing
discrete measures. Integration of the flow gradually assigns categories and
avoids issues of discretizing the latent continuous model like rounding, sample
truncation etc. General non-factorizing discrete distributions capable of
representing complex statistical dependencies of structured discrete data, can
be approximated by embedding the submanifold into a the meta-simplex of all
joint discrete distributions and data-driven averaging. Efficient training of
the generative model is demonstrated by matching the flow of geodesics of
factorizing discrete distributions. Various experiments underline the
approach's broad applicability.
