  Single-image piece-wise planar 3D reconstruction aims to simultaneously
segment plane instances and recover 3D plane parameters from an image. Most
recent approaches leverage convolutional neural networks (CNNs) and achieve
promising results. However, these methods are limited to detecting a fixed
number of planes with certain learned order. To tackle this problem, we propose
a novel two-stage method based on associative embedding, inspired by its recent
success in instance segmentation. In the first stage, we train a CNN to map
each pixel to an embedding space where pixels from the same plane instance have
similar embeddings. Then, the plane instances are obtained by grouping the
embedding vectors in planar regions via an efficient mean shift clustering
algorithm. In the second stage, we estimate the parameter for each plane
instance by considering both pixel-level and instance-level consistencies. With
the proposed method, we are able to detect an arbitrary number of planes.
Extensive experiments on public datasets validate the effectiveness and
efficiency of our method. Furthermore, our method runs at 30 fps at the testing
time, thus could facilitate many real-time applications such as visual SLAM and
human-robot interaction. Code is available at
https://github.com/svip-lab/PlanarReconstruction.
