  Slot-filling and intent detection are the backbone of conversational agents
such as voice assistants, and are active areas of research. Even though
state-of-the-art techniques on publicly available benchmarks show impressive
performance, their ability to generalize to realistic scenarios is yet to be
demonstrated. In this work, we present NATURE, a set of simple spoken-language
oriented transformations, applied to the evaluation set of datasets, to
introduce human spoken language variations while preserving the semantics of an
utterance. We apply NATURE to common slot-filling and intent detection
benchmarks and demonstrate that simple perturbations from the standard
evaluation set by NATURE can deteriorate model performance significantly.
Through our experiments we demonstrate that when NATURE operators are applied
to evaluation set of popular benchmarks the model accuracy can drop by up to
40%.
