  Humans ask follow-up questions driven by curiosity, which reflects a creative
human cognitive process. We introduce the task of real-world
information-seeking follow-up question generation (FQG), which aims to generate
follow-up questions seeking a more in-depth understanding of an initial
question and answer. We construct FOLLOWUPQG, a dataset of over 3K real-world
(initial question, answer, follow-up question) tuples collected from a Reddit
forum providing layman-friendly explanations for open-ended questions. In
contrast to existing datasets, questions in FOLLOWUPQG use more diverse
pragmatic strategies to seek information, and they also show higher-order
cognitive skills (such as applying and relating). We evaluate current question
generation models on their efficacy for generating follow-up questions,
exploring how to generate specific types of follow-up questions based on
step-by-step demonstrations. Our results validate FOLLOWUPQG as a challenging
benchmark, as model-generated questions are adequate but far from human-raised
questions in terms of informativeness and complexity.
