  Natural language provides a widely accessible and expressive interface for
robotic agents. To understand language in complex environments, agents must
reason about the full range of language inputs and their correspondence to the
world. Such reasoning over language and vision is an open problem that is
receiving increasing attention. While existing data sets focus on visual
diversity, they do not display the full range of natural language expressions,
such as counting, set reasoning, and comparisons.
  We propose a simple task for natural language visual reasoning, where images
are paired with descriptive statements. The task is to predict if a statement
is true for the given scene. This abstract describes our existing synthetic
images corpus and our current work on collecting real vision data.
