  The ability to recognize the position and order of the floor-level lines that
divide adjacent building floors can benefit many applications, for example,
urban augmented reality (AR). This work tackles the problem of locating
floor-level lines in street-view images, using a supervised deep learning
approach. Unfortunately, very little data is available for training such a
network $-$ current street-view datasets contain either semantic annotations
that lack geometric attributes, or rectified facades without perspective
priors. To address this issue, we first compile a new dataset and develop a new
data augmentation scheme to synthesize training samples by harassing (i) the
rich semantics of existing rectified facades and (ii) perspective priors of
buildings in diverse street views. Next, we design FloorLevel-Net, a multi-task
learning network that associates explicit features of building facades and
implicit floor-level lines, along with a height-attention mechanism to help
enforce a vertical ordering of floor-level lines. The generated segmentations
are then passed to a second-stage geometry post-processing to exploit
self-constrained geometric priors for plausible and consistent reconstruction
of floor-level lines. Quantitative and qualitative evaluations conducted on
assorted facades in existing datasets and street views from Google demonstrate
the effectiveness of our approach. Also, we present context-aware image overlay
results and show the potentials of our approach in enriching AR-related
applications.
