  Visual Grounding (VG) in Visual Question Answering (VQA) systems describes
how well a system manages to tie a question and its answer to relevant image
regions. Systems with strong VG are considered intuitively interpretable and
suggest an improved scene understanding. While VQA accuracy performances have
seen impressive gains over the past few years, explicit improvements to VG
performance and evaluation thereof have often taken a back seat on the road to
overall accuracy improvements. A cause of this originates in the predominant
choice of learning paradigm for VQA systems, which consists of training a
discriminative classifier over a predetermined set of answer options.
  In this work, we break with the dominant VQA modeling paradigm of
classification and investigate VQA from the standpoint of an information
retrieval task. As such, the developed system directly ties VG into its core
search procedure. Our system operates over a weighted, directed, acyclic graph,
a.k.a. "lattice", which is derived from the scene graph of a given image in
conjunction with region-referring expressions extracted from the question.
  We give a detailed analysis of our approach and discuss its distinctive
properties and limitations. Our approach achieves the strongest VG performance
among examined systems and exhibits exceptional generalization capabilities in
a number of scenarios.
