  We, as human beings, can understand and picture a familiar scene from
arbitrary viewpoints given a single image, whereas this is still a grand
challenge for computers. We hereby present a novel solution to mimic such human
perception capability based on a new paradigm of amodal 3D scene understanding
with neural rendering for a closed scene. Specifically, we first learn the
prior knowledge of the objects in a closed scene via an offline stage, which
facilitates an online stage to understand the room with unseen furniture
arrangement. During the online stage, given a panoramic image of the scene in
different layouts, we utilize a holistic neural-rendering-based optimization
framework to efficiently estimate the correct 3D scene layout and deliver
realistic free-viewpoint rendering. In order to handle the domain gap between
the offline and online stage, our method exploits compositional neural
rendering techniques for data augmentation in the offline training. The
experiments on both synthetic and real datasets demonstrate that our two-stage
design achieves robust 3D scene understanding and outperforms competing methods
by a large margin, and we also show that our realistic free-viewpoint rendering
enables various applications, including scene touring and editing. Code and
data are available on the project webpage:
https://zju3dv.github.io/nr_in_a_room/.
