  Acquiring a large vocabulary is an important aspect of human intelligence.
Onecommon approach for human to populating vocabulary is to learn words
duringreading or listening, and then use them in writing or speaking. This
ability totransfer from input to output is natural for human, but it is
difficult for machines.Human spontaneously performs this knowledge transfer in
complicated multimodaltasks, such as Visual Question Answering (VQA). In order
to approach human-levelArtificial Intelligence, we hope to equip machines with
such ability. Therefore, toaccelerate this research, we propose a newzero-shot
transfer VQA(ZST-VQA)dataset by reorganizing the existing VQA v1.0 dataset in
the way that duringtraining, some words appear only in one module (i.e.
questions) but not in theother (i.e. answers). In this setting, an intelligent
model should understand andlearn the concepts from one module (i.e. questions),
and at test time, transfer themto the other (i.e. predict the concepts as
answers). We conduct evaluation on thisnew dataset using three existing
state-of-the-art VQA neural models. Experimentalresults show a significant drop
in performance on this dataset, indicating existingmethods do not address the
zero-shot transfer problem. Besides, our analysis findsthat this may be caused
by the implicit bias learned during training.
