  This paper introduces the task of visual question answering for remote
sensing data (RSVQA). Remote sensing images contain a wealth of information
which can be useful for a wide range of tasks including land cover
classification, object counting or detection. However, most of the available
methodologies are task-specific, thus inhibiting generic and easy access to the
information contained in remote sensing data. As a consequence, accurate remote
sensing product generation still requires expert knowledge. With RSVQA, we
propose a system to extract information from remote sensing data that is
accessible to every user: we use questions formulated in natural language and
use them to interact with the images. With the system, images can be queried to
obtain high level information specific to the image content or relational
dependencies between objects visible in the images. Using an automatic method
introduced in this article, we built two datasets (using low and high
resolution data) of image/question/answer triplets. The information required to
build the questions and answers is queried from OpenStreetMap (OSM). The
datasets can be used to train (when using supervised methods) and evaluate
models to solve the RSVQA task. We report the results obtained by applying a
model based on Convolutional Neural Networks (CNNs) for the visual part and on
a Recurrent Neural Network (RNN) for the natural language part to this task.
The model is trained on the two datasets, yielding promising results in both
cases.
