  Probabilistic models based on continuous latent spaces, such as variational
autoencoders, can be understood as uncountable mixture models where components
depend continuously on the latent code. They have proven to be expressive tools
for generative and probabilistic modelling, but are at odds with tractable
probabilistic inference, that is, computing marginals and conditionals of the
represented probability distribution. Meanwhile, tractable probabilistic models
such as probabilistic circuits (PCs) can be understood as hierarchical discrete
mixture models, and thus are capable of performing exact inference efficiently
but often show subpar performance in comparison to continuous latent-space
models. In this paper, we investigate a hybrid approach, namely continuous
mixtures of tractable models with a small latent dimension. While these models
are analytically intractable, they are well amenable to numerical integration
schemes based on a finite set of integration points. With a large enough number
of integration points the approximation becomes de-facto exact. Moreover, for a
finite set of integration points, the integration method effectively compiles
the continuous mixture into a standard PC. In experiments, we show that this
simple scheme proves remarkably effective, as PCs learnt this way set new state
of the art for tractable models on many standard density estimation benchmarks.
