  Training dialogue systems often entails dealing with noisy training examples
and unexpected user inputs. Despite their prevalence, there currently lacks an
accurate survey of dialogue noise, nor is there a clear sense of the impact of
each noise type on task performance. This paper addresses this gap by first
constructing a taxonomy of noise encountered by dialogue systems. In addition,
we run a series of experiments to show how different models behave when
subjected to varying levels of noise and types of noise. Our results reveal
that models are quite robust to label errors commonly tackled by existing
denoising algorithms, but that performance suffers from dialogue-specific
noise. Driven by these observations, we design a data cleaning algorithm
specialized for conversational settings and apply it as a proof-of-concept for
targeted dialogue denoising.
