  This paper aims to tackle the challenge of efficiently producing interactive
free-viewpoint videos. Some recent works equip neural radiance fields with
image encoders, enabling them to generalize across scenes. When processing
dynamic scenes, they can simply treat each video frame as an individual scene
and perform novel view synthesis to generate free-viewpoint videos. However,
their rendering process is slow and cannot support interactive applications. A
major factor is that they sample lots of points in empty space when inferring
radiance fields. We propose a novel scene representation, called ENeRF, for the
fast creation of interactive free-viewpoint videos. Specifically, given
multi-view images at one frame, we first build the cascade cost volume to
predict the coarse geometry of the scene. The coarse geometry allows us to
sample few points near the scene surface, thereby significantly improving the
rendering speed. This process is fully differentiable, enabling us to jointly
learn the depth prediction and radiance field networks from RGB images.
Experiments on multiple benchmarks show that our approach exhibits competitive
performance while being at least 60 times faster than previous generalizable
radiance field methods.
