  This paper proposes CQ-VQA, a novel 2-level hierarchical but end-to-end model
to solve the task of visual question answering (VQA). The first level of
CQ-VQA, referred to as question categorizer (QC), classifies questions to
reduce the potential answer search space. The QC uses attended and fused
features of the input question and image. The second level, referred to as
answer predictor (AP), comprises of a set of distinct classifiers corresponding
to each question category. Depending on the question category predicted by QC,
only one of the classifiers of AP remains active. The loss functions of QC and
AP are aggregated together to make it an end-to-end model. The proposed model
(CQ-VQA) is evaluated on the TDIUC dataset and is benchmarked against
state-of-the-art approaches. Results indicate competitive or better performance
of CQ-VQA.
