  Visual Question Answering (VQA) systems are tasked with answering natural
language questions corresponding to a presented image. Traditional VQA datasets
typically contain questions related to the spatial information of objects,
object attributes, or general scene questions. Recently, researchers have
recognized the need to improve the balance of such datasets to reduce the
system's dependency on memorized linguistic features and statistical biases,
while aiming for enhanced visual understanding. However, it is unclear whether
any latent patterns exist to quantify and explain these failures. As an initial
step towards better quantifying our understanding of the performance of VQA
models, we use a taxonomy of Knowledge Gaps (KGs) to tag questions with one or
more types of KGs. Each Knowledge Gap (KG) describes the reasoning abilities
needed to arrive at a resolution. After identifying KGs for each question, we
examine the skew in the distribution of questions for each KG. We then
introduce a targeted question generation model to reduce this skew, which
allows us to generate new types of questions for an image. These new questions
can be added to existing VQA datasets to increase the diversity of questions
and reduce the skew.
