  Large-scale and high-quality corpora are necessary for evaluating machine
reading comprehension models on a low-resource language like Vietnamese.
Besides, machine reading comprehension (MRC) for the health domain offers great
potential for practical applications; however, there is still very little MRC
research in this domain. This paper presents ViNewsQA as a new corpus for the
Vietnamese language to evaluate healthcare reading comprehension models. The
corpus comprises 22,057 human-generated question-answer pairs. Crowd-workers
create the questions and their answers based on a collection of over 4,416
online Vietnamese healthcare news articles, where the answers comprise spans
extracted from the corresponding articles. In particular, we develop a process
of creating a corpus for the Vietnamese machine reading comprehension.
Comprehensive evaluations demonstrate that our corpus requires abilities beyond
simple reasoning, such as word matching and demanding difficult reasoning based
on single-or-multiple-sentence information. We conduct experiments using
different types of machine reading comprehension methods to achieve the first
baseline performances, compared with further models' performances. We also
measure human performance on the corpus and compared it with several powerful
neural network-based and transfer learning-based models. Our experiments show
that the best machine model is ALBERT, which achieves an exact match score of
65.26% and an F1-score of 84.89% on our corpus. The significant differences
between humans and the best-performance model (14.53% of EM and 10.90% of
F1-score) on the test set of our corpus indicate that improvements in ViNewsQA
could be explored in the future study. Our corpus is publicly available on our
website for the research purpose to encourage the research community to make
these improvements.
