  We present Worldsheet, a method for novel view synthesis using just a single
RGB image as input. The main insight is that simply shrink-wrapping a planar
mesh sheet onto the input image, consistent with the learned intermediate
depth, captures underlying geometry sufficient to generate photorealistic
unseen views with large viewpoint changes. To operationalize this, we propose a
novel differentiable texture sampler that allows our wrapped mesh sheet to be
textured and rendered differentiably into an image from a target viewpoint. Our
approach is category-agnostic, end-to-end trainable without using any 3D
supervision, and requires a single image at test time. We also explore a simple
extension by stacking multiple layers of Worldsheets to better handle
occlusions. Worldsheet consistently outperforms prior state-of-the-art methods
on single-image view synthesis across several datasets. Furthermore, this
simple idea captures novel views surprisingly well on a wide range of
high-resolution in-the-wild images, converting them into navigable 3D pop-ups.
Video results and code are available at https://worldsheet.github.io.
