  Webpages have been a rich, scalable resource for vision-language and language
only tasks. Yet only pieces of webpages are kept in existing datasets:
image-caption pairs, long text articles, or raw HTML, never all in one place.
Webpage tasks have resultingly received little attention and structured
image-text data left underused. To study multimodal webpage understanding, we
introduce the Wikipedia Webpage suite (WikiWeb2M) containing 2M pages with all
of the associated image, text, and structure data. We verify its utility on
three generative tasks: page description generation, section summarization, and
contextual image captioning. We design a novel attention mechanism Prefix
Global, which selects the most relevant image and text content as global tokens
to attend to the rest of the webpage for context. By using page structure to
separate such tokens, it performs better than full attention with lower
computational complexity. Extensive experiments show that the new data in
WikiWeb2M improves task performance compared to prior work.
