  Automated methods for essay scoring have made great progress in recent years,
achieving accuracies very close to human annotators. However, a known weakness
of such automated scorers is not taking into account the semantic relevance of
the submitted text. While there is existing work on detecting answer relevance
given a textual prompt, very little previous research has been done to
incorporate visual writing prompts. We propose a neural architecture and
several extensions for detecting off-topic responses to visual prompts and
evaluate it on a dataset of texts written by language learners.
