  Collection of annotated dialogs for training task-oriented dialog systems
have been one of the key bottlenecks in improving current models. While dialog
response generation has been widely studied on the agent side, it is not
evident if similar generative models can be used to generate a large variety
of, and often unexpected, user inputs that real dialog systems encounter in
practice. Existing data augmentation techniques such as paraphrase generation
do not take the dialog context into consideration. In this paper, we develop a
novel dialog augmentation model that generates a user turn, conditioning on
full dialog context. Additionally, with a new prompt design for language model,
and output re-ranking, the dialogs generated from our model can be directly
used to train downstream dialog systems. On common benchmark datasets MultiWoZ
and SGD, we show that our dialog augmentation model generates high quality
dialogs and improves dialog success rate by as much as $8\%$ over baseline.
