  Bridging the semantic gap between image and question is an important step to
improve the accuracy of the Visual Question Answering (VQA) task. However, most
of the existing VQA methods focus on attention mechanisms or visual relations
for reasoning the answer, while the features at different semantic levels are
not fully utilized. In this paper, we present a new reasoning framework to fill
the gap between visual features and semantic clues in the VQA task. Our method
first extracts the features and predicates from the image and question. We then
propose a new reasoning framework to effectively jointly learn these features
and predicates in a coarse-to-fine manner. The intensively experimental results
on three large-scale VQA datasets show that our proposed approach achieves
superior accuracy comparing with other state-of-the-art methods. Furthermore,
our reasoning framework also provides an explainable way to understand the
decision of the deep neural network when predicting the answer.
