  Neural Radiance Field (NeRF) has revolutionized free viewpoint rendering
tasks and achieved impressive results. However, the efficiency and accuracy
problems hinder its wide applications. To address these issues, we propose
Geometry-Aware Generalized Neural Radiance Field (GARF) with a geometry-aware
dynamic sampling (GADS) strategy to perform real-time novel view rendering and
unsupervised depth estimation on unseen scenes without per-scene optimization.
Distinct from most existing generalized NeRFs, our framework infers the unseen
scenes on both pixel-scale and geometry-scale with only a few input images.
More specifically, our method learns common attributes of novel-view synthesis
by an encoder-decoder structure and a point-level learnable multi-view feature
fusion module which helps avoid occlusion. To preserve scene characteristics in
the generalized model, we introduce an unsupervised depth estimation module to
derive the coarse geometry, narrow down the ray sampling interval to proximity
space of the estimated surface and sample in expectation maximum position,
constituting Geometry-Aware Dynamic Sampling strategy (GADS). Moreover, we
introduce a Multi-level Semantic Consistency loss (MSC) to assist more
informative representation learning. Extensive experiments on indoor and
outdoor datasets show that comparing with state-of-the-art generalized NeRF
methods, GARF reduces samples by more than 25\%, while improving rendering
quality and 3D geometry estimation.
