  Polygonal building outline extraction has been a research focus in recent
years. Most existing methods have addressed this challenging task by
decomposing it into several subtasks and employing carefully designed
architectures. Despite their accuracy, such pipelines often introduce
inefficiencies during training and inference. This paper presents an end-to-end
framework, denoted as PolyR-CNN, which offers an efficient and fully integrated
approach to predict vectorized building polygons and bounding boxes directly
from remotely sensed images. Notably, PolyR-CNN leverages solely the features
of the Region of Interest (RoI) for the prediction, thereby mitigating the
necessity for complex designs. Furthermore, we propose a novel scheme with
PolyR-CNN to extract detailed outline information from polygon vertex
coordinates, termed vertex proposal feature, to guide the RoI features to
predict more regular buildings. PolyR-CNN demonstrates the capacity to deal
with buildings with holes through a simple post-processing method on the Inria
dataset. Comprehensive experiments conducted on the CrowdAI dataset show that
PolyR-CNN achieves competitive accuracy compared to state-of-the-art methods
while significantly improving computational efficiency, i.e., achieving 79.2
Average Precision (AP), exhibiting a 15.9 AP gain and operating 2.5 times
faster and four times lighter than the well-established end-to-end method
PolyWorld. Replacing the backbone with a simple ResNet-50, PolyR-CNN maintains
a 71.1 AP while running four times faster than PolyWorld.
