  Polygonal building outlines are crucial for geographic and cartographic
applications. The existing approaches for outline extraction from aerial or
satellite imagery are typically decomposed into subtasks, e.g., building
masking and vectorization, or treat this task as a sequence-to-sequence
prediction of ordered vertices. The former lacks efficiency, and the latter
often generates redundant vertices, both resulting in suboptimal performance.
To handle these issues, we propose a novel Region-of-Interest (RoI) query-based
approach called RoIPoly. Specifically, we formulate each vertex as a query and
constrain the query attention on the most relevant regions of a potential
building, yielding reduced computational overhead and more efficient vertex
level interaction. Moreover, we introduce a novel learnable logit embedding to
facilitate vertex classification on the attention map; thus, no post-processing
is needed for redundant vertex removal. We evaluated our method on the
vectorized building outline extraction dataset CrowdAI and the 2D floorplan
reconstruction dataset Structured3D. On the CrowdAI dataset, RoIPoly with a
ResNet50 backbone outperforms existing methods with the same or better
backbones on most MS-COCO metrics, especially on small buildings, and achieves
competitive results in polygon quality and vertex redundancy without any
post-processing. On the Structured3D dataset, our method achieves the
second-best performance on most metrics among existing methods dedicated to 2D
floorplan reconstruction, demonstrating our cross-domain generalization
capability. The code will be released upon acceptance of this paper.
