  Talking head generation based on the neural radiation fields model has shown
promising visual effects. However, the slow rendering speed of NeRF seriously
limits its application, due to the burdensome calculation process over hundreds
of sampled points to synthesize one pixel. In this work, a novel Neural Light
Dynamic Fields model is proposed aiming to achieve generating high quality 3D
talking face with significant speedup. The NLDF represents light fields based
on light segments, and a deep network is used to learn the entire light beam's
information at once. In learning the knowledge distillation is applied and the
NeRF based synthesized result is used to guide the correct coloration of light
segments in NLDF. Furthermore, a novel active pool training strategy is
proposed to focus on high frequency movements, particularly on the speaker
mouth and eyebrows. The propose method effectively represents the facial light
dynamics in 3D talking video generation, and it achieves approximately 30 times
faster speed compared to state of the art NeRF based method, with comparable
generation visual quality.
