  We present $\textbf{$\texttt{SkillQG}$}$: a question generation framework
with controllable comprehension types for assessing and improving machine
reading comprehension models. Existing question generation systems widely
differentiate questions by $\textit{literal}$ information such as question
words and answer types to generate semantically relevant questions for a given
context. However, they rarely consider the $\textit{comprehension}$ nature of
questions, i.e. the different comprehension capabilities embodied by different
questions. In comparison, our $\texttt{SkillQG}$ is able to tailor a
fine-grained assessment and improvement to the capabilities of question
answering models built on it. Specifically, we first frame the comprehension
type of questions based on a hierarchical skill-based schema, then formulate
$\texttt{SkillQG}$ as a skill-conditioned question generator. Furthermore, to
improve the controllability of generation, we augment the input text with
question focus and skill-specific knowledge, which are constructed by
iteratively prompting the pre-trained language models. Empirical results
demonstrate that $\texttt{SkillQG}$ outperforms baselines in terms of quality,
relevance, and skill-controllability while showing a promising performance
boost in downstream question answering task.
