  To date, most work on text simplification has focused on sentence-level
inputs. Early attempts at document simplification merely applied these
approaches iteratively over the sentences of a document. However, this fails to
coherently preserve the discourse structure, leading to suboptimal output
quality. Recently, strategies from controllable simplification have been
leveraged to achieve state-of-the-art results on document simplification by
first generating a document-level plan (a sequence of sentence-level
simplification operations) and using this plan to guide sentence-level
simplification downstream. However, this is still limited in that the
simplification model has no direct access to the local inter-sentence document
context, likely having a negative impact on surface realisation. We explore
various systems that use document context within the simplification process
itself, either by iterating over larger text units or by extending the system
architecture to attend over a high-level representation of document context. In
doing so, we achieve state-of-the-art performance on the document
simplification task, even when not relying on plan-guidance. Further, we
investigate the performance and efficiency tradeoffs of system variants and
make suggestions of when each should be preferred.
