  Alternatively inferring on the visual facts and commonsense is fundamental
for an advanced VQA system. This ability requires models to go beyond the
literal understanding of commonsense. The system should not just treat objects
as the entrance to query background knowledge, but fully ground commonsense to
the visual world and imagine the possible relationships between objects, e.g.,
"fork, can lift, food". To comprehensively evaluate such abilities, we propose
a VQA benchmark, CRIC, which introduces new types of questions about
Compositional Reasoning on vIsion and Commonsense, and an evaluation metric
integrating the correctness of answering and commonsense grounding. To collect
such questions and rich additional annotations to support the metric, we also
propose an automatic algorithm to generate question samples from the scene
graph associated with the images and the relevant knowledge graph. We further
analyze several representative types of VQA models on the CRIC dataset.
Experimental results show that grounding the commonsense to the image region
and joint reasoning on vision and commonsense are still challenging for current
approaches. The dataset is available at https://cricvqa.github.io.
