  Reconstructing 3D layouts from multiple $360^{\circ}$ panoramas has received
increasing attention recently as estimating a complete layout of a large-scale
and complex room from a single panorama is very difficult. The state-of-the-art
method, called PSMNet, introduces the first learning-based framework that
jointly estimates the room layout and registration given a pair of panoramas.
However, PSMNet relies on an approximate (i.e., "noisy") registration as input.
Obtaining this input requires a solution for wide baseline registration which
is a challenging problem. In this work, we present a complete multi-view
panoramic layout estimation framework that jointly learns panorama registration
and layout estimation given a pair of panoramas without relying on a pose
prior. The major improvement over PSMNet comes from a novel Geometry-aware
Panorama Registration Network or GPR-Net that effectively tackles the wide
baseline registration problem by exploiting the layout geometry and computing
fine-grained correspondences on the layout boundaries, instead of the global
pixel-space. Our architecture consists of two parts. First, given two
panoramas, we adopt a vision transformer to learn a set of 1D horizon features
sampled on the panorama. These 1D horizon features encode the depths of
individual layout boundary samples and the correspondence and covisibility maps
between layout boundaries. We then exploit a non-linear registration module to
convert these 1D horizon features into a set of corresponding 2D boundary
points on the layout. Finally, we estimate the final relative camera pose via
RANSAC and obtain the complete layout simply by taking the union of registered
layouts. Experimental results indicate that our method achieves
state-of-the-art performance in both panorama registration and layout
estimation on a large-scale indoor panorama dataset ZInD.
