  Existing approaches to system identification (estimating the physical
parameters of an object) from videos assume known object geometries. This
precludes their applicability in a vast majority of scenes where object
geometries are complex or unknown. In this work, we aim to identify parameters
characterizing a physical system from a set of multi-view videos without any
assumption on object geometry or topology. To this end, we propose "Physics
Augmented Continuum Neural Radiance Fields" (PAC-NeRF), to estimate both the
unknown geometry and physical parameters of highly dynamic objects from
multi-view videos. We design PAC-NeRF to only ever produce physically plausible
states by enforcing the neural radiance field to follow the conservation laws
of continuum mechanics. For this, we design a hybrid Eulerian-Lagrangian
representation of the neural radiance field, i.e., we use the Eulerian grid
representation for NeRF density and color fields, while advecting the neural
radiance fields via Lagrangian particles. This hybrid Eulerian-Lagrangian
representation seamlessly blends efficient neural rendering with the material
point method (MPM) for robust differentiable physics simulation. We validate
the effectiveness of our proposed framework on geometry and physical parameter
estimation over a vast range of materials, including elastic bodies,
plasticine, sand, Newtonian and non-Newtonian fluids, and demonstrate
significant performance gain on most tasks.
