  Urban facade segmentation from automatically acquired imagery, in contrast to
traditional image segmentation, poses several unique challenges. 360-degree
photospheres captured from vehicles are an effective way to capture a large
number of images, but this data presents difficult-to-model warping and
stitching artifacts. In addition, each pixel can belong to multiple facade
elements, and different facade elements (e.g., window, balcony, sill, etc.) are
correlated and vary wildly in their characteristics. In this paper, we propose
three network architectures of varying complexity to achieve multilabel
semantic segmentation of facade images while exploiting their unique
characteristics. Specifically, we propose a MULTIFACSEGNET architecture to
assign multiple labels to each pixel, a SEPARABLE architecture as a low-rank
formulation that encourages extraction of rectangular elements, and a
COMPATIBILITY network that simultaneously seeks segmentation across facade
element types allowing the network to 'see' intermediate output probabilities
of the various facade element classes. Our results on benchmark datasets show
significant improvements over existing facade segmentation approaches for the
typical facade elements. For example, on one commonly used dataset, the
accuracy scores for window(the most important architectural element) increases
from 0.91 to 0.97 percent compared to the best competing method, and comparable
improvements on other element types.
