  We introduce RNADE, a new model for joint density estimation of real-valued
vectors. Our model calculates the density of a datapoint as the product of
one-dimensional conditionals modeled using mixture density networks with shared
parameters. RNADE learns a distributed representation of the data, while having
a tractable expression for the calculation of densities. A tractable likelihood
allows direct comparison with other methods and training by standard
gradient-based optimizers. We compare the performance of RNADE on several
datasets of heterogeneous and perceptual data, finding it outperforms mixture
models in all but one case.
