  In recent years, neural implicit representations gained popularity in 3D
reconstruction due to their expressiveness and flexibility. However, the
implicit nature of neural implicit representations results in slow inference
time and requires careful initialization. In this paper, we revisit the classic
yet ubiquitous point cloud representation and introduce a differentiable
point-to-mesh layer using a differentiable formulation of Poisson Surface
Reconstruction (PSR) that allows for a GPU-accelerated fast solution of the
indicator function given an oriented point cloud. The differentiable PSR layer
allows us to efficiently and differentiably bridge the explicit 3D point
representation with the 3D mesh via the implicit indicator field, enabling
end-to-end optimization of surface reconstruction metrics such as Chamfer
distance. This duality between points and meshes hence allows us to represent
shapes as oriented point clouds, which are explicit, lightweight and
expressive. Compared to neural implicit representations, our Shape-As-Points
(SAP) model is more interpretable, lightweight, and accelerates inference time
by one order of magnitude. Compared to other explicit representations such as
points, patches, and meshes, SAP produces topology-agnostic, watertight
manifold surfaces. We demonstrate the effectiveness of SAP on the task of
surface reconstruction from unoriented point clouds and learning-based
reconstruction.
