  We study the problem of object detection over scanned images of scientific
documents. We consider images that contain objects of varying aspect ratios and
sizes and range from coarse elements such as tables and figures to fine
elements such as equations and section headers. We find that current object
detectors fail to produce properly localized region proposals over such page
objects. We revisit the original R-CNN model and present a method for
generating fine-grained proposals over document elements. We also present a
region embedding model that uses the convolutional maps of a proposal's
neighbors as context to produce an embedding for each proposal. This region
embedding is able to capture the semantic relationships between a target region
and its surrounding context. Our end-to-end model produces an embedding for
each proposal, then classifies each proposal by using a multi-head attention
model that attends to the most important neighbors of a proposal. To evaluate
our model, we collect and annotate a dataset of publications from heterogeneous
journals. We show that our model, referred to as Attentive-RCNN, yields a 17%
mAP improvement compared to standard object detection models.
