  We propose CodeQA, a free-form question answering dataset for the purpose of
source code comprehension: given a code snippet and a question, a textual
answer is required to be generated. CodeQA contains a Java dataset with 119,778
question-answer pairs and a Python dataset with 70,085 question-answer pairs.
To obtain natural and faithful questions and answers, we implement syntactic
rules and semantic analysis to transform code comments into question-answer
pairs. We present the construction process and conduct systematic analysis of
our dataset. Experiment results achieved by several neural baselines on our
dataset are shown and discussed. While research on question-answering and
machine reading comprehension develops rapidly, few prior work has drawn
attention to code question answering. This new dataset can serve as a useful
research benchmark for source code comprehension.
