  Triangular map is a recent construct in probability theory that allows one to
transform any source probability density function to any target density
function. Based on triangular maps, we propose a general framework for
high-dimensional density estimation, by specifying one-dimensional
transformations (equivalently conditional densities) and appropriate
conditioner networks. This framework (a) reveals the commonalities and
differences of existing autoregressive and flow based methods, (b) allows a
unified understanding of the limitations and representation power of these
recent approaches and, (c) motivates us to uncover a new Sum-of-Squares (SOS)
flow that is interpretable, universal, and easy to train. We perform several
synthetic experiments on various density geometries to demonstrate the benefits
(and short-comings) of such transformations. SOS flows achieve competitive
results in simulations and several real-world datasets.
