  Finding a good way to model probability densities is key to probabilistic
inference. An ideal model should be able to concisely approximate any
probability while being also compatible with two main operations:
multiplications of two models (product rule) and marginalization with respect
to a subset of the random variables (sum rule). In this work, we show that a
recently proposed class of positive semi-definite (PSD) models for non-negative
functions is particularly suited to this end. In particular, we characterize
both approximation and generalization capabilities of PSD models, showing that
they enjoy strong theoretical guarantees. Moreover, we show that we can perform
efficiently both sum and product rule in closed form via matrix operations,
enjoying the same versatility of mixture models. Our results open the way to
applications of PSD models to density estimation, decision theory and
inference.
