  We introduce KFD-NeRF, a novel dynamic neural radiance field integrated with
an efficient and high-quality motion reconstruction framework based on Kalman
filtering. Our key idea is to model the dynamic radiance field as a dynamic
system whose temporally varying states are estimated based on two sources of
knowledge: observations and predictions. We introduce a novel plug-in Kalman
filter guided deformation field that enables accurate deformation estimation
from scene observations and predictions. We use a shallow Multi-Layer
Perceptron (MLP) for observations and model the motion as locally linear to
calculate predictions with motion equations. To further enhance the performance
of the observation MLP, we introduce regularization in the canonical space to
facilitate the network's ability to learn warping for different frames.
Additionally, we employ an efficient tri-plane representation for encoding the
canonical space, which has been experimentally demonstrated to converge quickly
with high quality. This enables us to use a shallower observation MLP,
consisting of just two layers in our implementation. We conduct experiments on
synthetic and real data and compare with past dynamic NeRF methods. Our
KFD-NeRF demonstrates similar or even superior rendering performance within
comparable computational time and achieves state-of-the-art view synthesis
performance with thorough training.
