  We present Virtual Elastic Objects (VEOs): virtual objects that not only look
like their real-world counterparts but also behave like them, even when subject
to novel interactions. Achieving this presents multiple challenges: not only do
objects have to be captured including the physical forces acting on them, then
faithfully reconstructed and rendered, but also plausible material parameters
found and simulated. To create VEOs, we built a multi-view capture system that
captures objects under the influence of a compressed air stream. Building on
recent advances in model-free, dynamic Neural Radiance Fields, we reconstruct
the objects and corresponding deformation fields. We propose to use a
differentiable, particle-based simulator to use these deformation fields to
find representative material parameters, which enable us to run new
simulations. To render simulated objects, we devise a method for integrating
the simulation results with Neural Radiance Fields. The resulting method is
applicable to a wide range of scenarios: it can handle objects composed of
inhomogeneous material, with very different shapes, and it can simulate
interactions with other virtual objects. We present our results using a newly
collected dataset of 12 objects under a variety of force fields, which will be
shared with the community.
