  Code generation from text requires understanding the user's intent from a
natural language description and generating an executable code snippet that
satisfies this intent. While recent pretrained language models demonstrate
remarkable performance for this task, these models fail when the given natural
language description is under-specified. In this work, we introduce a novel and
more realistic setup for this task. We hypothesize that the under-specification
of a natural language description can be resolved by asking clarification
questions. Therefore, we collect and introduce a new dataset named CodeClarQA
containing pairs of natural language descriptions and code with created
synthetic clarification questions and answers. The empirical results of our
evaluation of pretrained language model performance on code generation show
that clarifications result in more precisely generated code, as shown by the
substantial improvement of model performance in all evaluation metrics.
Alongside this, our task and dataset introduce new challenges to the community,
including when and what clarification questions should be asked. Our code and
dataset are available on GitHub.
