  HTML documents are an important medium for disseminating information on the
Web for human consumption. An HTML document presents information in multiple
text formats including unstructured text, structured key-value pairs, and
tables. Effective representation of these documents is essential for machine
understanding to enable a wide range of applications, such as Question
Answering, Web Search, and Personalization. Existing work has either
represented these documents using visual features extracted by rendering them
in a browser, which is typically computationally expensive, or has simply
treated them as plain text documents, thereby failing to capture useful
information presented in their HTML structure. We argue that the text and HTML
structure together convey important semantics of the content and therefore
warrant a special treatment for their representation learning. In this paper,
we introduce a novel representation learning approach for web pages, dubbed
DOM-LM, which addresses the limitations of existing approaches by encoding both
text and DOM tree structure with a transformer-based encoder and learning
generalizable representations for HTML documents via self-supervised
pre-training. We evaluate DOM-LM on a variety of webpage understanding tasks,
including Attribute Extraction, Open Information Extraction, and Question
Answering. Our extensive experiments show that DOM-LM consistently outperforms
all baselines designed for these tasks. In particular, DOM-LM demonstrates
better generalization performance both in few-shot and zero-shot settings,
making it attractive for making it suitable for real-world application settings
with limited labeled data.
