  Humans have a strong intuitive understanding of physical processes such as
fluid falling by just a glimpse of such a scene picture, i.e., quickly derived
from our immersive visual experiences in memory. This work achieves such a
photo-to-fluid-dynamics reconstruction functionality learned from unannotated
videos, without any supervision of ground-truth fluid dynamics. In a nutshell,
a differentiable Euler simulator modeled with a ConvNet-based pressure
projection solver, is integrated with a volumetric renderer, supporting
end-to-end/coherent differentiable dynamic simulation and rendering. By
endowing each sampled point with a fluid volume value, we derive a NeRF-like
differentiable renderer dedicated from fluid data; and thanks to this
volume-augmented representation, fluid dynamics could be inversely inferred
from the error signal between the rendered result and ground-truth video frame
(i.e., inverse rendering). Experiments on our generated Fluid Fall datasets and
DPI Dam Break dataset are conducted to demonstrate both effectiveness and
generalization ability of our method.
