  Light Field Networks, the re-formulations of radiance fields to oriented
rays, are magnitudes faster than their coordinate network counterparts, and
provide higher fidelity with respect to representing 3D structures from 2D
observations. They would be well suited for generic scene representation and
manipulation, but suffer from one problem: they are limited to holistic and
static scenes. In this paper, we propose the Dynamic Light Field Network
(DyLiN) method that can handle non-rigid deformations, including topological
changes. We learn a deformation field from input rays to canonical rays, and
lift them into a higher dimensional space to handle discontinuities. We further
introduce CoDyLiN, which augments DyLiN with controllable attribute inputs. We
train both models via knowledge distillation from pretrained dynamic radiance
fields. We evaluated DyLiN using both synthetic and real world datasets that
include various non-rigid deformations. DyLiN qualitatively outperformed and
quantitatively matched state-of-the-art methods in terms of visual fidelity,
while being 25 - 71x computationally faster. We also tested CoDyLiN on
attribute annotated data and it surpassed its teacher model. Project page:
https://dylin2023.github.io .
