  Whole slide imaging is routinely adopted for carcinoma diagnosis and
prognosis. Abundant experience is required for pathologists to achieve accurate
and reliable diagnostic results of whole slide images (WSI). The huge size and
heterogeneous features of WSIs make the workflow of pathological reading
extremely time-consuming. In this paper, we propose a novel framework (WSI-VQA)
to interpret WSIs by generative visual question answering. WSI-VQA shows
universality by reframing various kinds of slide-level tasks in a
question-answering pattern, in which pathologists can achieve
immunohistochemical grading, survival prediction, and tumor subtyping following
human-machine interaction. Furthermore, we establish a WSI-VQA dataset which
contains 8672 slide-level question-answering pairs with 977 WSIs. Besides the
ability to deal with different slide-level tasks, our generative model which is
named Wsi2Text Transformer (W2T) outperforms existing discriminative models in
medical correctness, which reveals the potential of our model to be applied in
the clinical scenario. Additionally, we also visualize the co-attention mapping
between word embeddings and WSIs as an intuitive explanation for diagnostic
results. The dataset and related code are available at
https://github.com/cpystan/WSI-VQA.
