  Predicting panoramic indoor lighting from a single perspective image is a
fundamental but highly ill-posed problem in computer vision and graphics. To
achieve locale-aware and robust prediction, this problem can be decomposed into
three sub-tasks: depth-based image warping, panorama inpainting and
high-dynamic-range (HDR) reconstruction, among which the success of panorama
inpainting plays a key role. Recent methods mostly rely on convolutional neural
networks (CNNs) to fill the missing contents in the warped panorama. However,
they usually achieve suboptimal performance since the missing contents occupy a
very large portion in the panoramic space while CNNs are plagued by limited
receptive fields. The spatially-varying distortion in the spherical signals
further increases the difficulty for conventional CNNs. To address these
issues, we propose a local-to-global strategy for large-scale panorama
inpainting. In our method, a depth-guided local inpainting is first applied on
the warped panorama to fill small but dense holes. Then, a transformer-based
network, dubbed PanoTransformer, is designed to hallucinate reasonable global
structures in the large holes. To avoid distortion, we further employ cubemap
projection in our design of PanoTransformer. The high-quality panorama
recovered at any locale helps us to capture spatially-varying indoor
illumination with physically-plausible global structures and fine details.
