  Learning from free-text human feedback is essential for dialog systems, but
annotated data is scarce and usually covers only a small fraction of error
types known in conversational AI. Instead of collecting and annotating new
datasets from scratch, recent advances in synthetic dialog generation could be
used to augment existing dialog datasets with the necessary annotations.
However, to assess the feasibility of such an effort, it is important to know
the types and frequency of free-text human feedback included in these datasets.
In this work, we investigate this question for a variety of commonly used
dialog datasets, including MultiWoZ, SGD, BABI, PersonaChat,
Wizards-of-Wikipedia, and the human-bot split of the Self-Feeding Chatbot.
Using our observations, we derive new taxonomies for the annotation of
free-text human feedback in dialogs and investigate the impact of including
such data in response generation for three SOTA language generation models,
including GPT-2, LLAMA, and Flan-T5. Our findings provide new insights into the
composition of the datasets examined, including error types, user response
types, and the relations between them.
