  Natural reading orders of words are crucial for information extraction from
form-like documents. Despite recent advances in Graph Convolutional Networks
(GCNs) on modeling spatial layout patterns of documents, they have limited
ability to capture reading orders of given word-level node representations in a
graph. We propose Reading Order Equivariant Positional Encoding (ROPE), a new
positional encoding technique designed to apprehend the sequential presentation
of words in documents. ROPE generates unique reading order codes for
neighboring words relative to the target word given a word-level graph
connectivity. We study two fundamental document entity extraction tasks
including word labeling and word grouping on the public FUNSD dataset and a
large-scale payment dataset. We show that ROPE consistently improves existing
GCNs with a margin up to 8.4% F1-score.
