  Capturing and rendering novel views of complex real-world scenes is a
long-standing problem in computer graphics and vision, with applications in
augmented and virtual reality, immersive experiences and 3D photography. The
advent of deep learning has enabled revolutionary advances in this area,
classically known as image-based rendering. However, previous approaches
require intractably dense view sampling or provide little or no guidance for
how users should sample views of a scene to reliably render high-quality novel
views. Local light field fusion proposes an algorithm for practical view
synthesis from an irregular grid of sampled views that first expands each
sampled view into a local light field via a multiplane image scene
representation, then renders novel views by blending adjacent local light
fields. Crucially, we extend traditional plenoptic sampling theory to derive a
bound that specifies precisely how densely users should sample views of a given
scene when using our algorithm. We achieve the perceptual quality of Nyquist
rate view sampling while using up to 4000x fewer views. Subsequent developments
have led to new scene representations for deep learning with view synthesis,
notably neural radiance fields, but the problem of sparse view synthesis from a
small number of images has only grown in importance. We reprise some of the
recent results on sparse and even single image view synthesis, while posing the
question of whether prescriptive sampling guidelines are feasible for the new
generation of image-based rendering algorithms.
