  We propose Pulsar, an efficient sphere-based differentiable renderer that is
orders of magnitude faster than competing techniques, modular, and easy-to-use
due to its tight integration with PyTorch. Differentiable rendering is the
foundation for modern neural rendering approaches, since it enables end-to-end
training of 3D scene representations from image observations. However,
gradient-based optimization of neural mesh, voxel, or function representations
suffers from multiple challenges, i.e., topological inconsistencies, high
memory footprints, or slow rendering speeds. To alleviate these problems,
Pulsar employs: 1) a sphere-based scene representation, 2) an efficient
differentiable rendering engine, and 3) neural shading. Pulsar executes orders
of magnitude faster than existing techniques and allows real-time rendering and
optimization of representations with millions of spheres. Using spheres for the
scene representation, unprecedented speed is obtained while avoiding topology
problems. Pulsar is fully differentiable and thus enables a plethora of
applications, ranging from 3D reconstruction to general neural rendering.
