  This article introduces AI2D-RST, a multimodal corpus of 1000
English-language diagrams that represent topics in primary school natural
sciences, such as food webs, life cycles, moon phases and human physiology. The
corpus is based on the Allen Institute for Artificial Intelligence Diagrams
(AI2D) dataset, a collection of diagrams with crowd-sourced descriptions, which
was originally developed to support research on automatic diagram understanding
and visual question answering. Building on the segmentation of diagram layouts
in AI2D, the AI2D-RST corpus presents a new multi-layer annotation schema that
provides a rich description of their multimodal structure. Annotated by trained
experts, the layers describe (1) the grouping of diagram elements into
perceptual units, (2) the connections set up by diagrammatic elements such as
arrows and lines, and (3) the discourse relations between diagram elements,
which are described using Rhetorical Structure Theory (RST). Each annotation
layer in AI2D-RST is represented using a graph. The corpus is freely available
for research and teaching.
