  During task-oriented dialogues (TODs), human users naturally introduce
chitchat that is beyond the immediate scope of the task, interfering with the
flow of the conversation. To address this issue without the need for expensive
manual data creation, we use few-shot prompting with Llama-2-70B to enhance the
MultiWOZ dataset with user backstories, a typical example of chitchat
interference in TODs. We assess the impact of this addition by testing two
models: one trained solely on TODs and another trained on TODs with a
preliminary chitchat interaction. Our analysis demonstrates that our enhanced
dataset poses a challenge for these systems. Moreover, we demonstrate that our
dataset can be effectively used for training purposes, enabling a system to
consistently acknowledge the user's backstory while also successfully moving
the task forward in the same turn, as confirmed by human evaluation. These
findings highlight the benefits of generating novel chitchat-TOD scenarios to
test TOD systems more thoroughly and improve their resilience to natural user
interferences
