  The internal workings of modern deep learning models stay often unclear to an
external observer, although spatial attention mechanisms are involved. The idea
of this work is to translate these spatial attentions into natural language to
provide a simpler access to the model's function. Thus, I took a neural image
captioning model and measured the reactions to external modification in its
spatial attention for three different interface methods: a fixation over the
whole generation process, a fixation for the first time-steps and an addition
to the generator's attention. The experimental results for bounding box based
spatial attention vectors have shown that the captioning model reacts to method
dependent changes in up to 52.65% and includes in 9.00% of the cases object
categories, which were otherwise unmentioned. Afterwards, I established such a
link to a hierarchical co-attention network for visual question answering by
extraction of its word, phrase and question level spatial attentions. Here,
generated captions for the word level included details of the question-answer
pairs in up to 55.20% of the cases. This work indicates that spatial attention
seen as an external interface for image caption generators is an useful method
to access visual functions in natural language.
