  Question Answering (QA) has shown great success thanks to the availability of
large-scale datasets and the effectiveness of neural models. Recent research
works have attempted to extend these successes to the settings with few or no
labeled data available. In this work, we introduce two approaches to improve
unsupervised QA. First, we harvest lexically and syntactically divergent
questions from Wikipedia to automatically construct a corpus of question-answer
pairs (named as RefQA). Second, we take advantage of the QA model to extract
more appropriate answers, which iteratively refines data over RefQA. We conduct
experiments on SQuAD 1.1, and NewsQA by fine-tuning BERT without access to
manually annotated data. Our approach outperforms previous unsupervised
approaches by a large margin and is competitive with early supervised models.
We also show the effectiveness of our approach in the few-shot learning
setting.
