  For many business applications that require the processing, indexing, and
retrieval of professional documents such as legal briefs (in PDF format etc.),
it is often essential to classify the pages of any given document into their
corresponding types beforehand. Most existing studies in the field of document
image classification either focus on single-page documents or treat multiple
pages in a document independently. Although in recent years a few techniques
have been proposed to exploit the context information from neighboring pages to
enhance document page classification, they typically cannot be utilized with
large pre-trained language models due to the constraint on input length. In
this paper, we present a simple but effective approach that overcomes the above
limitation. Specifically, we enhance the input with extra tokens carrying
sequential information about previous pages - introducing recurrence - which
enables the usage of pre-trained Transformer models like BERT for context-aware
page classification. Our experiments conducted on two legal datasets in English
and Portuguese respectively show that the proposed approach can significantly
improve the performance of document page classification compared to the
non-recurrent setup as well as the other context-aware baselines.
