  We introduce FigureQA, a visual reasoning corpus of over one million
question-answer pairs grounded in over 100,000 images. The images are
synthetic, scientific-style figures from five classes: line plots, dot-line
plots, vertical and horizontal bar graphs, and pie charts. We formulate our
reasoning task by generating questions from 15 templates; questions concern
various relationships between plot elements and examine characteristics like
the maximum, the minimum, area-under-the-curve, smoothness, and intersection.
To resolve, such questions often require reference to multiple plot elements
and synthesis of information distributed spatially throughout a figure. To
facilitate the training of machine learning systems, the corpus also includes
side data that can be used to formulate auxiliary objectives. In particular, we
provide the numerical data used to generate each figure as well as bounding-box
annotations for all plot elements. We study the proposed visual reasoning task
by training several models, including the recently proposed Relation Network as
a strong baseline. Preliminary results indicate that the task poses a
significant machine learning challenge. We envision FigureQA as a first step
towards developing models that can intuitively recognize patterns from visual
representations of data.
