  Point set is a flexible and lightweight representation widely used for 3D
deep learning. However, their discrete nature prevents them from representing
continuous and fine geometry, posing a major issue for learning-based shape
generation. In this work, we turn the discrete point sets into smooth surfaces
by introducing the well-known implicit moving least-squares (IMLS) surface
formulation, which naturally defines locally implicit functions on point sets.
We incorporate IMLS surface generation into deep neural networks for inheriting
both the flexibility of point sets and the high quality of implicit surfaces.
Our IMLSNet predicts an octree structure as a scaffold for generating MLS
points where needed and characterizes shape geometry with learned local priors.
Furthermore, our implicit function evaluation is independent of the neural
network once the MLS points are predicted, thus enabling fast runtime
evaluation. Our experiments on 3D object reconstruction demonstrate that
IMLSNets outperform state-of-the-art learning-based methods in terms of
reconstruction quality and computational efficiency. Extensive ablation tests
also validate our network design and loss functions.
