  3D shape reconstruction is a primary component of augmented/virtual reality.
Despite being highly advanced, existing solutions based on RGB, RGB-D and Lidar
sensors are power and data intensive, which introduces challenges for
deployment in edge devices. We approach 3D reconstruction with an event camera,
a sensor with significantly lower power, latency and data expense while
enabling high dynamic range. While previous event-based 3D reconstruction
methods are primarily based on stereo vision, we cast the problem as multi-view
shape from silhouette using a monocular event camera. The output from a moving
event camera is a sparse point set of space-time gradients, largely sketching
scene/object edges and contours. We first introduce an event-to-silhouette
(E2S) neural network module to transform a stack of event frames to the
corresponding silhouettes, with additional neural branches for camera pose
regression. Second, we introduce E3D, which employs a 3D differentiable
renderer (PyTorch3D) to enforce cross-view 3D mesh consistency and fine-tune
the E2S and pose network. Lastly, we introduce a 3D-to-events simulation
pipeline and apply it to publicly available object datasets and generate
synthetic event/silhouette training pairs for supervised learning.
