  Point clouds are a popular representation for 3D shapes. However, they encode
a particular sampling without accounting for shape priors or non-local
information. We advocate for the use of a hierarchical Gaussian mixture model
(hGMM), which is a compact, adaptive and lightweight representation that
probabilistically defines the underlying 3D surface. We present PointGMM, a
neural network that learns to generate hGMMs which are characteristic of the
shape class, and also coincide with the input point cloud. PointGMM is trained
over a collection of shapes to learn a class-specific prior. The hierarchical
representation has two main advantages: (i) coarse-to-fine learning, which
avoids converging to poor local-minima; and (ii) (an unsupervised) consistent
partitioning of the input shape. We show that as a generative model, PointGMM
learns a meaningful latent space which enables generating consistent
interpolations between existing shapes, as well as synthesizing novel shapes.
We also present a novel framework for rigid registration using PointGMM, that
learns to disentangle orientation from structure of an input shape.
