  Deep neural networks have been critical in the task of Visual Question
Answering (VQA), with research traditionally focused on improving model
accuracy. Recently, however, there has been a trend towards evaluating the
robustness of these models against adversarial attacks. This involves assessing
the accuracy of VQA models under increasing levels of noise in the input, which
can target either the image or the proposed query question, dubbed the main
question. However, there is currently a lack of proper analysis of this aspect
of VQA. This work proposes a new method that utilizes semantically related
questions, referred to as basic questions, acting as noise to evaluate the
robustness of VQA models. It is hypothesized that as the similarity of a basic
question to the main question decreases, the level of noise increases. To
generate a reasonable noise level for a given main question, a pool of basic
questions is ranked based on their similarity to the main question, and this
ranking problem is cast as a LASSO optimization problem. Additionally, this
work proposes a novel robustness measure, R_score, and two basic question
datasets to standardize the analysis of VQA model robustness. The experimental
results demonstrate that the proposed evaluation method effectively analyzes
the robustness of VQA models. Moreover, the experiments show that in-context
learning with a chain of basic questions can enhance model accuracy.
