  We introduce a technique for the reconstruction of high-fidelity surfaces
from multi-view images. Our technique uses a new point-based representation,
the dipole sum, which generalizes the winding number to allow for interpolation
of arbitrary per-point attributes in point clouds with noisy or outlier points.
Using dipole sums allows us to represent implicit geometry and radiance fields
as per-point attributes of a point cloud, which we initialize directly from
structure from motion. We additionally derive Barnes-Hut fast summation schemes
for accelerated forward and reverse-mode dipole sum queries. These queries
facilitate the use of ray tracing to efficiently and differentiably render
images with our point-based representations, and thus update their point
attributes to optimize scene geometry and appearance. We evaluate this inverse
rendering framework against state-of-the-art alternatives, based on ray tracing
of neural representations or rasterization of Gaussian point-based
representations. Our technique significantly improves reconstruction quality at
equal runtimes, while also supporting more general rendering techniques such as
shadow rays for direct illumination. In the supplement, we provide interactive
visualizations of our results.
